Apparatus for multiple camera devices and method of operating same

ABSTRACT

There are many, many inventions described herein. In one aspect, what is disclosed is a digital camera including a plurality of arrays of photo detectors, including a first array of photo detectors to sample an intensity of light of a first wavelength and a second array of photo detectors to sample an intensity of light of a second wavelength. The digital camera further may also include a first lens disposed in an optical path of the first array of photo detectors, wherein the first lens includes a predetermined optical response to the light of the first wavelength, and a second lens disposed in with an optical path of the second array of photo detectors wherein the second lens includes a predetermined optical response to the light of the second wavelength. In addition, the digital camera may include signal processing circuitry, coupled to the first and second arrays of photo detectors, to generate a composite image using (i) data which is representative of the intensity of light sampled by the first array of photo detectors, and (ii) data which is representative of the intensity of light sampled by the second array of photo detectors; wherein the first array of photo detectors, the second array of photo detectors, and the signal processing circuitry are integrated on or in the same semiconductor substrate.

RELATED APPLICATIONS

This application claims priority to: (1) U.S. Provisional ApplicationSer. No. 60/604,854, entitled “Solid State Camera”, filed Aug. 25, 2004;and (2) U.S. Provisional Application Ser. No. 60/695,946, entitled“Method and Apparatus for use in Camera and Systems Employing Same”,filed Jul. 1, 2005 (collectively “the Provisional Applications”). Thecontents of the Provisional Applications are incorporated by referenceherein in their entirety.

FIELD OF THE INVENTION

The field of the invention is digital imaging.

BACKGROUND

The recent technology transition from film to “electronic media” hasspurred the rapid growth of the imaging industry with applicationsincluding still and video cameras, cell phones, other personalcommunications devices, surveillance equipment, automotive applications,computer based video communication and conferencing, manufacturing andinspection devices, medical appliances, toys, plus a wide range of otherand continuously expanding applications. The lower cost and size ofdigital cameras (whether as stand-alone products or imbedded in otherappliances) is a primary driver for this growth and market expansion.

Although traditional component manufacturers continue to shrink thecomponents to take advantage of the electronic media, it is difficult toachieve the ever tightening demand of digital camera producers forsmaller sizes, lower costs and higher performance. Several importantissues remain, including: 1) the smaller the size of a digital camera(e.g., in cell phones), the poorer the image quality; 2) complex“lenses”, shutter and flash are still required for medium to higherquality imaging, thus negating much of the size advantage afforded bythe electronic media; and 3) the cost advantage afforded by theelectronic media is somewhat negated by the traditional complex andcostly lens systems and other peripheral components.

Most applications are continuously looking for all or some combinationof higher performance (image quality), features, smaller size and/orlower cost. These market needs can often be in conflict: higherperformance often requires larger size, improved features can requirehigher cost as well as a larger size, and conversely, reduced costand/or size can come at a penalty in performance and/or features. As anexample, consumers look for higher quality images from their cellphones, but are unwilling to accept the size or cost associated withputting stand-alone digital camera quality into their pocket sizedphones.

One driver to this challenge is the lens system for digital cameras. Asthe number of photo-detectors (pixels) increases, which increases imageresolution, the lenses must become larger to span the increased size ofthe image sensor which carries the photo detectors. The pixel size canbe reduced to maintain a constant image sensor and optics size as thenumber of pixels increases but pixel performance is reduced (reducedphoto-signal and increased crosstalk between pixels). Also, thedesirable “zoom lens” feature adds additional moveable opticalcomponents, size and cost to a lens system. Zoom, as performed by thelens system, known as “optical zoom”, changes the focal length of theoptics and is a highly desired feature. These attributes (for example,increased number of pixels in the image sensor and optical zoom),although benefiting image quality and features, may adversely impact thecamera size and cost. In some cases, such as cell phones or otherappliances where size and/or cost are critical, this approach to goodimage quality (high resolution and sensitivity) is not optimum.

Digital camera suppliers have one advantage over traditional filmproviders in the area of zoom capability. Through electronic processing,digital cameras can provide “electronic zoom” which provides the zoomcapability by cropping the outer regions of an image and thenelectronically enlarging the center region to the original size of theimage. In a manner similar to traditional enlargements, a degree ofresolution is lost when performing this process. Further, since digitalcameras capture discrete input to form a picture rather than theubiquitous process of film, the lost resolution is more pronounced. Assuch, although “electronic zoom” is a desired feature, it is not adirect substitute for “optical zoom.”

Conventional digital cameras typically use a single aperture and lenssystem to image the scene onto one or more image sensors. Colorseparation (if desired), such as red, green and blue (RGB), is typicallyachieved by three methods: 1) a color filter array on a singleintegrated circuit image sensor, 2) multiple image sensors with a colorseparation means in the optical path (such as prisms), or 3) an imagerwith color separation and multiple signal collection capability withineach pixel. These three color separation method have limitations asnoted below.

The color filter array, such as the often used Bayer pattern, changesthe incident color between adjacent pixels on the array and colorcrosstalk occurs that prevents accurate color rendition of the originalimage. Since the array is populated with pixels of different colorcapability, interpolation techniques are required to create a suitablecolor image. The color filter array may also have low and variableoptical transmission that reduces received optical signal levels andcreates pixel-to-pixel image non-uniformity.

The use of multiple imagers, with color separation methods such as aprism, provides accurate color rendition but the optical assembly islarge and expensive.

Color separation methods within the pixel create crosstalk of colors andinaccurate color rendition. Since multiple color charge collection andreadout means are required in each pixel, pixel size reduction islimited.

Technology advances in lenslet optical design and fabrication,integrated circuit imager pixel size reduction and digitalpost-processing have opened new possibilities for cameras and imagingsystems which differ dramatically in form fit and function fromtime-honored digital camera designs. The use of multiple camera channels(multiple optics, image sensors and electronics) in a compact assemblyallows fabrication of a digital camera with improved image quality,reduced physical thickness and increased imaging functionality notachievable with a conventional single aperture/optical system digitalcamera architecture.

SUMMARY OF THE INVENTION(S)

It should be understood that there are many inventions described andillustrated herein. Indeed, the present invention is neither limited toany single aspect nor embodiment thereof, nor to any combinations and/orpermutations of such aspects and/or embodiments. Moreover, each of theaspects of the present invention, and/or embodiments thereof, may beemployed alone or in combination with one or more of the other aspectsof the present invention and/or embodiments thereof. For the sake ofbrevity, many of those permutations and combinations will not bediscussed separately herein.

In one aspect of the present invention, an image sensor comprisesseparate first and second arrays of photo detectors and a signalprocessing circuitry that combines signals from the arrays to produce acomposite image.

Preferred embodiments include three or more arrays of photo detectors,wherein the signal processing circuitry processes the signals from eacharray and then combines the signals from all of the arrays to produce acomposite image. Such use of multiple arrays allows each of the arraysto be optimized in some respect, such as for receipt of particularcolors. Thus, for example, the arrays can be optimized to detect lightof different colors, or other wavelengths. The “colors” can be narrowbands, or broad bands such as red, green, or blue. The bands can even beoverlapping.

Optimization can be accomplished in any desired manner, including forexample having different average pixel depths, column logic, analogsignal logic, black level logic, exposure control, image processingtechniques, and lens design and coloration.

A sensor having two or more different arrays could advantageously have adifferent lens over each of the different arrays. Preferred lenses canemploy a die coating, defused dye in the optical medium, a substantiallyuniform color filter or any other filtering technique through whichlight passes to the underlying array.

The processing circuitry can comprise any suitable mechanism and/orlogic. Of particular interest are circuitries that produce multipleseparate images from the different arrays, and then combines themultiple separate images to form a single image. During the process thesignal processing circuitry can advantageously execute image enhancementfunctions, such as address saturation, sharpness, intensity, hue,artifact removal, and defective pixel correction.

As far as integration, it is desirable for the various arrays to bephysically located on the same chip. In addition, it is desirable tocouple a frame to the chip, and to couple at least one of the lenses tothe frame. The lenses can be independently positionable duringmanufacture, and then sealed to the frame using a sealant or otherbonding technique. The integration of these elements is called the“Digital Camera Subsystem” (DCS).

Preferred image sensors contain at least several hundred thousand of thephoto detectors, and have a total thickness of no more than 10, 15, or20 mm, including the lens and frame. Such small DCS devices may beincorporated into a semiconductor “package” or directly attached to acircuit board (“packageless”), using wave soldering, die on board, orother techniques. The DCS and/or board can then be incorporated intocameras or other devices having user interface elements, memory thatstores images derived from the arrays, and at least one power supplythat provides power to the system. The DCS, cameras and other devices ofthe invention can be used for any suitable purpose, including especiallystill and video imaging, calculating a distance, and creating a 3Deffect.

In another aspect of the present invention a compact solid-state camera(compact digital camera) comprises a first and second camera channel,located in close proximity to each other, where each camera channelcontains its own optics, image sensor and signal processing circuitry.The two camera channels (being identical or different) can combine theiroutput signals to form a composite image or each camera channel canprovide a separate image. The electronics to combine images from anycombination of camera channels or to display/store/transmit channelsindividually or combined is included in the compact solid-state cameraassembly (CSSC).

Other embodiments include three or more camera channels (identical ordifferent), wherein the signal processing circuitry processes thesignals from each channel and then combines the signals from some or allthe channels to produce a composite image or each camera channel canprovide a separate image by itself in conjunction with a compositeimage. The use of multiple camera channels allows each of the channelsto be optimized in some respect, if desired, such as imaging ofparticular incident light colors. Thus, for example, the arrays can beoptimized to detect light of different colors, or other wavelengths. The“colors” of each camera channel can be narrow bands, or broad bands suchas red, green, or blue. The bands can even be overlapping. Each camerachannel can image one or more colors.

Optimization of each camera channel can be accomplished in any desiredmanner, including optics, image sensor and signal processing electronicsto obtain a desired image capability, for example the optics canoptimized for a certain image sensor size, wavelength (color), focallength and f-number. The image sensor can be optimized by number ofpixels, pixel size, pixel design (photo-detector and circuitry), framerate, integration time, and peripheral circuitry external to the pixelcircuitry. The signal processing electronics can be optimized for colorcorrection, image compression, bad pixel replacement and other imagingfunctions. The camera channels can be identical or unique; however alllocated in close proximity.

Color filters (or other color separation techniques), if desired, can beincorporated into the optical materials or optics surfaces, as separatefilter layers, on the image sensor surface or built into the pixelsemiconductor by design. Each camera channel can have its own colorimaging characteristics. The image sensor can have a single colorcapability or multiple color capability; this multiple color capabilitycan be within a single pixel, or between adjacent pixels (orcombinations of single and adjacent pixels)

The processing circuitry can comprise any suitable mechanism and/orlogic to optimize the image quality. Of particular interest arecircuitries that produce separate images from the camera channels, andthen combines the multiple separate images to form a composite singleimage. During the process, the signal processing circuitry canadvantageously execute image enhancement functions, such as dynamicrange management (auto gain/level), image sharpening, intensitycorrection, hue, artifact removal, defective pixel correction and otherimaging optimization functions. The processing circuitry can operate inan analog or digital mode.

As far as mechanical integration, it is desirable for the various imagesensors of the camera channels be physically located on the sameintegrated circuit (chip) to reduce manufacturing costs and reduceelectrical interconnects and size. In addition, it is desirable toassemble a mechanical frame to the chip, and to couple one or more ofthe lenses to the frame. The lenses can be independently positionableduring manufacture, and then sealed to the frame using a sealant orother bonding technique. The integration of these elements is called the“Digital Camera Subsystem” (DCS). The vertical integration of otherlayers to the DCS (such as camera system electronics and even displaycapability), can form a compact solid-state camera (CSSC)

Preferred camera channels contain at least several hundred thousand ofthe photo detectors (pixels). The thickness of the camera channels(including image sensors and optics) can be thinner than conventionalcamera systems (for equivalent image resolution) where only one opticalassembly is utilized. Such small DCS devices may be incorporated into asemiconductor “package” or directly attached to a circuit board(“packageless”), using wave soldering, die on board, or othertechniques. The DCS and/or board can then be incorporated into camerasor other devices having user interface elements, memory that storesimages derived from the arrays, and at least one power supply thatprovides power to the system. The DCS, cameras and other devices of theinvention can be used for any suitable purpose, including especiallystill and video imaging.

Notably, in certain aspects, a digital camera subassembly includes twoor more complete camera channels in a single layered assembly thatcontains all desired components (optics, mechanical structures andelectronics) in one heterogeneous assembly or package.

In another embodiment, the digital camera subassembly has the form of amulti-layer laminate.

In another embodiment, two or more of the camera channels includeschannel specific optics, optical alignment structures (mechanicalframe), packaging, color filters and other optical elements, imagesensors, mixed signal interface, image and/or color processing logic,memory, control and timing logic, power management logic and paralleland/or serial device interface.

In another embodiment, each camera channel also includes one or more ofthe following: single or multi-channel image compression logic and/orimage output formatting logic, wired or wireless communications, andoptical display capability.

In another embodiment, the output of each channel can provide eitherdiscrete processed images or integrated images comprising a color orpartial color images.

In another embodiment, the camera channels are co-located, in closeproximity defined by number of, type of, and position of, and opticaldiameter constraints of the lens system, on a two-dimensional focalplane that comprises one component layer of the CSSC.

In another embodiment, each camera channel further contains an imagesensor to provide a photon sensing capability that makes up part of theoverall compact solid-state camera using semiconductor-based detectionmechanisms (no film). The single assembly may be formed by two or morecomponent layers that are assembled sequentially in the verticaldimension (orthogonal to the focal plane).

In another embodiment, the assembly, comprising the verticallyintegrated component layers, with multiple camera channel capability,provides camera system capability and performance not achievable withconventional camera systems using a single camera channel.

In another embodiment, some or the entire vertically integratedcomponent layers are be formed by methods of wafer scale integration orlaminated assembly to create portions of many camera systemssimultaneously.

In another embodiment, the wafers or layers may contain optical,mechanical and electrical components, electrical interconnects and otherdevices (such as a display).

In another embodiment, the electrical interconnect between componentlayers may be formed by lithography and metallization, bump bonding orother methods. Organic or inorganic bonding methods can be used to jointhe component layers. The layered assembly process starts with a “host”wafer with electronics used for the entire camera and/or each camerachannel. Then another wafer or individual chips are aligned and bondedto the host wafer. The transferred wafers or chips can have bumps tomake electrical interconnect or connects can be made after bonding andthinning. The support substrate from the second wafer or individualchips is removed, leaving only a few microns material thickness attachedto the host wafer containing the transferred electronics. Electricalinterconnects are then made (if needed) between the host and the bondedwafer or die using standard integrated circuit processes. The processcan be repeated multiple times. The layers transferred in this fashioncan contain electrical, mechanical or optical features/components. Thisprocess allows multiple layers to form a heterogeneous assembly withelectrical, mechanical and optical capabilities required in a compactsolid-state camera.

In another embodiment, the camera channels are comprised of linear orarea array imagers, of any size, format, pixel number, pixel design orpixel pitch.

In another embodiment, the camera channels provide full color, singlecolor, multi-color or mono chromatic (black and white) capability in anywavelength range from ultraviolet (UV) to infrared (IR). Color filters,if desired, may be on an image sensor or within the optical componentlayer or a combination of both. The camera channels may also providecolor capability by utilizing the semiconductor absorption properties ina pixel. For example, a pixel may provide one or more color capabilityvia the optical absorption depth properties. The pixel color separationproperties may also be combined with color filters in the optical path.

In another embodiment, a high spatial image resolution may be achievedby using multiple camera channels to observe the same field of view froma slightly different perspective.

In another embodiment, two or more camera channels observe the samefield of view, although from a different perspective as a result of aspatial offset between such camera channels. In some of suchembodiments, images from such two or more camera channels may becombined to result in an image that provides high spatial resolution. Itmay be advantageous to employ a parallax correction algorithm in orderto reduce and/or eliminate the effects of the parallax. Alternatively,images from the two or more camera channels (with the same field of viewbut different perspectives) may be combined to provide three dimensionalfeature imaging. In this regard, it may be advantageous to increaseand/or enhance the effects of the parallax, for example, by applying aparallax correction algorithm “inversely”. Three dimensional featureimaging may be used, for example, in finger print and/or retinal featureimaging and/or analysis. Any parallax correction algorithm whether nowknown or later developed may be employed in conjunction with any of theembodiments herein. Any of the previous embodiments can be employed inassociation with an increase in parallax and/or a decrease in parallax.

In another embodiment, optical features may be added to the opticalstack of one or more camera channels to provide additional imagingcapability such as single, dual or tunable color filters, wave frontmodification for increased depth of focus and auto focus, and glarereduction polarization filters. Notably, any optical feature whether nowknown or later developed may be incorporated in one or more of thecamera channels to provide additional imaging capability.

In another embodiment, the optical portion may include one or morefilters, e.g., color filters, to provide one or more wavelengths or oneor more bands of wavelengths to one or more associated sensor arrays.Such filters may be for example, single or dual, fixed or tunablefilters. In one embodiment, the user, operator and/or manufacturer mayemploy a tunable filter to control or determine the one or morewavelengths or the one or more bands of wavelengths.

In another embodiment, one or more filters are employed in associationwith one, some or all of the camera channels. Such filters may or maynot be the same as one another. For example, the filters may or may notprovide the same wavelength or bands of wavelengths. In addition, someof the filters may be fixed and others may be tunable.

In another embodiment, the optical portion includes a wave frontmodification element, for example, to increase the depth of focus and/orfor use in implementing auto focus. In addition, in another embodiment,the optical portion may include one or more glare reduction polarizationfilters, to polarize the light and thereby reduce “glare”. Such filtersmay be employed alone or in combination with any of the embodimentsdisclosed herein.

Any of the embodiments of the present invention may include one or moreillumination units to improve and/or enhance image acquisition by theone or more camera channels (and, in particular, the one or more sensorarrays), facilitate range detection to an object, shape detection of anobject, and covert imaging (i.e., imaging that is not observable to thehuman eye).

The illumination units may provide passive (for example, noillumination), active (for example, constant illumination), constantand/or gated active illumination (for example, pulsed illumination thatis predetermined, preset or processor controlled, and/or pulsedillumination that is user/operator programmable). The one or moreillumination units may be disposed on or integrated in the support frameand/or the substrate of the sensor arrays. Indeed, the one or moreillumination units may be disposed on or integrated in any element orcomponent of the one or more of the camera channels.

In some embodiments, the illumination units are dedicated to one or morecamera channels. In this regard, the illumination unit is “enabled”cooperatively with the operation of one or more dedicated channels. Inanother embodiment, the illumination units are shared by all of thecamera channels. As such, in this embodiment, the illumination unit isenabled cooperatively with the operation of the camera channels. Indeed,in certain embodiments, one or more illumination units may be dedicatedto one or more camera channels and one or more illumination units may beshared by one or more camera channels (including those channels that areassociated with one or more dedicated illumination units). In thisembodiment, the dedicated illumination units are “enabled” cooperativelywith the operation of one or more dedicated channels and the sharedillumination units are enabled cooperatively with the operation of allof the camera channels.

As noted above, one or more of the camera channels may be optimized,modified and/or configured according to a predetermined, an adaptivelydetermined, an anticipated and/or a desired spectral response of the oneor more camera channels. For example, the dimensions, characteristics,operation, response and/or parameters of a sensor array (and/or pixelsthereof) as well as image processing circuitry may be configured,designed and/or tailored in accordance with predetermined, adaptivelydetermined, anticipated and/or desired spectral response of the one ormore camera channels. In this way, one or more aspects of the digitalcamera of the present inventions may be configured, designed and/ortailored to provide a desired, suitable, predetermined and/or particularresponse in the environment in which the camera is to be employed.

In some embodiments, each camera channel may be uniquely configured,designed and/or tailored. For example, one or more camera channels maybe configured to include a field of view that is different than one ormore camera channels. As such, one or more camera channels have a firstfield of view and one or more other camera channels have a second fieldof view. In this way, a digital camera may simultaneously capture animage using different fields of view.

The field of view may be fixed or programmable (for example, in situ).The field of view may be adjusted using a number of techniques orconfigurations including adjustment or modification of the optics focallength and/or adjustment or modification of the effective size of thearray. Indeed, any technique or configuration to adjust the field ofview, whether now known or later developed, is intended to come withinthe scope of the present inventions.

Further, the digital camera of the present inventions may includeprogrammable (in situ or otherwise) or fixed integration times for oneor more (or all) of the camera channels. In this regard, integrationtime of one or more camera channels may be configured, designed and/ortailored to facilitate capture of, for example, a large scene dynamicrange. As such, in this embodiment, single color band camera channelsmay be used to create a combined color image capability (including, forexample, UV and IR if desired), configuring and/or designing theintegration time of each camera channel to provide desired signalcollection it its wavelength acquisition band.

Moreover, two or more integration times can be implemented tosimultaneously acquire low to high light levels in the image. Thecombined dynamic range of the multiple camera channels provides greaterdynamic range than from a single camera channel (having a oneintegration time for all channels). As such, the image sensor(s) orarray(s) of each camera channel may be configured and/or designed tooperate using a specific (predetermined, pre-set or programmable)integration time range and illumination level.

Notably, the dimensions, characteristics, operation, response and/orparameters of the camera channels (for example, field of view,integration time, sensor array (and/or pixels thereof), and/or imageprocessing circuitry) may be configured, designed and/or tailored (insitu or otherwise) in accordance with predetermined, an adaptivelydetermined, anticipated and/or desired response of the one or morecamera channels. For example, the camera channels may be configured,designed and/or tailored to include different fields of view each havingthe same or different frame rates and/or integration times. As such, inone embodiment, the digital camera of the present inventions may includea first large/wide field of view camera channel capability to acquireobjects and a second narrower field of view camera channel to identifyobjects. Moreover, the resolution of the first large/wide field of viewcamera channel and second narrower field of view camera channel may alsobe different in order to, for example, provide an enhanced image oracquisition.

In addition, the sensor array and/or pixel size (pitch) may beconfigured, designed and/or tailored in accordance with a predetermined,an anticipated and/or a desired response of the one or more camerachannels. For example, the pixel size may be configured in order tooptimize, enhance and/or obtain a particular response. In oneembodiment, the pixel size of associated sensor arrays may be selectedin order to provide, enhance and/or optimize a particular response ofthe digital camera. In this regard, where the sensor array includes aplurality of camera channels (for example, UV, B, R, G and IR),implementing different pixel sizes in one or more of the sensor arrays(for example, an increasing pixel pitch from UV (smallest) to IR(largest)) may provide, enhance and/or optimize a particular response ofthe digital camera.

The size of the pixels may be based on a number of considerationsincluding providing a predetermined, adaptively determined, anticipatedor a desired resolution and/or obtaining a predetermined, enhancedand/or suitable acquisition characteristics for certain wavelength orbands of wavelengths) for example, reducing the size of the pixel(reducing the size of the pitch) may enhance the acquisition of shorterwavelengths of light. This may be advantageous when matching thecorresponding reduction in optical blur size. The pixel design andprocess sequence (a subset of the total wafer process) may be selectedand/or determined to optimize and/or enhance photo-response of aparticular camera channel color. Moreover, the number of pixels on thesensor array may be adjusted, selected and/or determined to provide thesame field of view notwithstanding different sizes of the pixel in theplurality of arrays.

Further, the image processing circuitry (for example, the imageprocessing and color processing logic) may be configured, designedand/or tailored to provide a predetermined, an adaptively determined, ananticipated and/or a desired response of the one or more camerachannels. For example, the image processing and color processing logicmay be configured to optimize, accelerate and/or reduce the complexityby “matching” the optics, sensor, and image processing when discretelyapplied to each channel separately. Any final sequencing of a full orpartial color image may, in turn, be simplified and quality greatlyimproved via the elimination of Bayer pattern interpolation.

It should be noted that any of the digital camera channels (for exampleRGB capable or other color filter combinations) may be combined with oneor more full color, dual color, single color or B/W camera channels. Thecombination of camera channels may be used to provide increasedwavelength range capability, different simultaneous fields of view,different simultaneous integrations times, active and passive imagingcapability, higher resolution using multiple camera channels andparallax correction, 3D imaging (feature extraction) using multiplecamera channels and increased parallax, and increased color bandcapability.

In some embodiments, different color camera channels share components,for example, data processing components. In this regard, in oneembodiment, one camera channel may employ a sensor array that acquiresdata which is representative of a first color image (for example, blue)as well as a second color image (green). Other camera channels mayemploy sensor arrays that are dedicated to a particular/predeterminedwavelength or band of wavelengths (for example, red or green) or suchcamera channels may employ sensor arrays that acquires data which isrepresentative of two or more predetermined wavelengths or bands ofwavelengths (for example, (i) red and green or (ii) cyan and green). Thecamera channels, in combination, may provide full color capabilities.

For example, in one embodiment, a first sensor array may acquire datawhich is representative of first and second predetermined wavelengths orband of wavelengths (for example, wavelengths that are associated withred and blue) and a second sensor array may acquire data which isrepresentative of third predetermined wavelength or band of wavelengths(for example, wavelengths that are associated with green). In thisembodiment, the camera channels, in combination, may provide a fullcolor image using only two sensor array.

Notably, in the exemplary embodiment discussed above, it may beadvantageous to employ a third sensor array to acquire IR. In this way,a “true” YCrCb output camera may be provided while minimizing and/oreliminating the cost complexity and/or power considerations necessary toperform the transformation in the digital image domain.

Where a sensor array acquires two or more predetermined wavelengths orbands of wavelengths, the pixels of the sensor array may be designed tocollect photons at two or more depths or areas within the pixels of thesemiconductor arrays which are associated with the two or morepredetermined wavelengths or bands of wavelengths. In this regard, thecolor “selection” for such a sensor array may be based on color bandseparation and/or pixel design to color separate by optical absorptiondepth.

Moreover, the two color capability in one or more camera channel may beaccomplished or provided using color filter arrays that are disposedbefore the sensor array (for example, in the optical assembly). Notably,additional color band separation can be provided in the optical assemblylayer if desired.

It may be advantageous to employ programmable (in situ or otherwise) orfixed integration techniques for one or more (or all) of the camerachannels in conjunction with a sensor array that acquires two or morepredetermined or adaptively determined wavelengths or bands ofwavelengths. In this regard, integration time(s) of one or more camerachannels may be configured, designed and/or tailored to facilitatecapture of, for example, multiple predetermined wavelengths or bands ofwavelengths in order to enhance, optimize and or provide an enhanced,designed, desired adaptively determined and/or predetermined acquisitiontechnique. Notably, any of the embodiments discussed herein in relationto integration times of the camera channels may be incorporated with asensor array that acquires two or more predetermined wavelengths orbands of wavelength. For the sake of brevity, those discussions will notbe repeated here.

The present inventions may be implemented using three sensor arrays(each acquiring one or more predetermined wavelengths or band ofwavelengths for example, wavelengths that are associated with red, blueand green). In this embodiment, the three sensor arrays may be arrangedin a triangular configuration (for example, a symmetrical,non-symmetrical, isosceles, obtuse, acute and/or right triangle) toprovide full color (RGB) capability. The triangular configuration willprovide symmetry in parallax and thereby simplify the algorithmcomputation to address parallax. The triangular configuration will alsoprovide enhanced and/or optimal layout of a three image sensor arraysystem/device and associated assembly layers for a more compactassembly.

In the triangular configuration/layout embodiment, it may beadvantageous to employ programmable (in situ or otherwise) or fixedintegration techniques for one or more (or all) of the camera channels.In this regard, integration time of one or more camera channels may beconfigured, designed and/or tailored to facilitate capture of, forexample, multiple predetermined wavelengths or bands of wavelengths inorder to enhance, optimize and or provide an enhanced, desired, designedadaptive determined and/or predetermined acquisition techniques.Notably, any of the embodiments discussed above herein in relation tointegration times of the camera channels may be incorporated withtriangular configuration/layout. For the sake of brevity, thosediscussions will not be repeated here.

As mentioned above, the digital camera according to the presentinventions may include two or more camera channels. In one embodiment,the digital camera includes a plurality of sensor arrays (for example,greater than five sensor arrays) each acquiring a narrow predeterminednumber of wavelengths or band of wavelengths (for example, wavelengthsthat are associated with four to ten color bands). In this way, thedigital camera may provide multi-spectral (for example, 4-10 colorbands) or hyper-spectral (for example, 10-100 color bands) simultaneousimaging capability.

In another embodiment, the digital camera may employ black and white(B/W) sensor arrays that acquire multiple broadband B/W images. Thecombination of B/W camera channels may be used to provide increasedwavelength range capability, different simultaneous fields of view,different simultaneous integrations times, active and passive imagingcapability, higher resolution using multiple camera channels andparallax correction, 3D imaging (feature extraction) using multiplecamera channels and increased parallax. Indeed, the multiple B/W camerachannels can be combined with other camera channels for full or partialcolor capability. Notably, gray scale sensor arrays may be employed inconjunction with or in lieu of the B/W sensor arrays described herein.

In another embodiment, the digital camera subsystem includes a display.The display may be disposed in a display layer and/or integrated in oron the sensor array substrate.

In yet another embodiment, the digital camera subsystem provides one ormore interfaces for communicating with the digital camera subsystem.

In another embodiment, the digital camera subsystem includes thecapability for wired, wireless and/or optical communication. In someembodiments, the digital camera subsystem includes one or more circuits,or portions thereof, for use in such communication. The circuits may bedisposed in a layer dedicated for use in such communication and/or maybe incorporated into one of the other layers (for example, integrated inor on the sensor array substrate).

In one aspect of the present invention, a “scene” is imaged ontomultiple sensor arrays. The sensor arrays may be in close proximity andmay be processed on a single integrated circuit or fabricatedindividually and assembled close together. Each sensor array is locatedin or beneath an optical assembly. The optical assembly, can beprocessed of the sensor subsystem wafer, applied to the image wafer by aseparate wafer transfer, transferred individually by pick and placemethod, or attached at die level.

Where color filters are employed, the color filters can be built intothe optical material, disposed as a layer or coating on the associatedsensor array, applied as a lens coating or as a separate color filter inthe optical assembly. Color separation mechanisms can also be providedon each imaging area by means of color filters or by an in-pixel colorseparation mechanism if desired. Other optical features can be added tothe optical system of each sensor array to provide additional imagingcapability.

In some embodiments, the design and electrical operation of each sensorarray is optimized for sensing the incident wavelengths of light to thatsensor array. The use of multiple optical assemblies with individuallyoptimized sensor arrays results is a compact camera capable of highresolution, high sensitivity and excellent color rendition.

In one aspect, the present invention is a digital camera comprising aplurality of arrays of photo detectors, including a first array of photodetectors to sample an intensity of light of, for example, light of afirst wavelength (which may be associated with a first color) and asecond array of photo detectors to sample an intensity of light of, forexample, light of a second wavelength (which may be associated with asecond color). The digital camera may include signal processingcircuitry, coupled to the first and second arrays of photo detectors, togenerate a composite image using (i) data which is representative of theintensity of light sampled by the first array of photo detectors, and(ii) data which is representative of the intensity of light sampled bythe second array of photo detectors. In this aspect of the presentinvention, the first array of photo detectors, the second array of photodetectors, and the signal processing circuitry are integrated on or inthe same semiconductor substrate.

The digital camera may further include a third array of photo detectorsto sample the intensity of light of a third wavelength(which may beassociated with a third color). In this embodiment, the signalprocessing circuitry is coupled to the third array of photo detectorsand generates a composite image using (i) data which is representativeof the intensity of light sampled by the first array of photo detectors,(ii) data which is representative of the intensity of light sampled bythe second array of photo detectors, and (ii) data which isrepresentative of the intensity of light sampled by the third array ofphoto detectors. The first, second and third arrays of photo detectorsmay be relatively arranged in a triangular configuration (for example,an isosceles, obtuse, acute or a right triangular configuration).

In certain embodiments, the first array of photo detectors may samplethe intensity of light of the first wavelength for a first integrationtime; the second array of photo detectors sample the intensity of lightof the second wavelength for a second integration time. Where thedigital camera includes a third array of photo detectors, the thirdarray of photo detectors sample the intensity of light of the thirdwavelength for the first integration time, the second integration time,or a third integration time.

The digital camera may include a first array wherein each photo detectorof the first array includes a semiconductor portion at which theintensity of light is sampled. Further, each photo detector of thesecond array includes a semiconductor portion at which the intensity oflight is sampled. In certain embodiments, the semiconductor portion ofeach photo detector of the first array is located at a different depth,relative to a surface of each of the photo detectors, from thansemiconductor portion of each photo detector of the second array.

The digital camera may further include a first lens disposed in andassociated with an optical path of the first array of photo detectors aswell as a second lens disposed in and associated with an optical path ofthe second array of photo detectors. A substantially uniform colorfilter sheet may be disposed in the optical path of the first array ofdetectors. Further, a first colored lens disposed in and associated withan optical path of the first array of detectors.

Notably, the digital camera may further including a first lens (passeslight of a first wavelength and filters light of a second wavelength)disposed in and associated with an optical path of the first array ofphoto detectors, wherein the first array of photo detectors sample anintensity of light of a first wavelength and the second array of photodetectors sample an intensity of light of a second wavelength.

The digital camera may include a first array of photo detectors thatsamples an intensity of light of a first wavelength and an intensity oflight of a second wavelength and a second array of photo detectorssample an intensity of light of a third wavelength, wherein the firstwavelength is associated with a first color, the second wavelength isassociated with a second color and the third wavelength is associatedwith a third color. Each photo detector of the first array may include afirst semiconductor portion at which the intensity of light of the firstwavelength is sampled and a second semiconductor portion at which theintensity of light of the second wavelength is sampled; and each photodetector of the second array may include a semiconductor portion atwhich the intensity of light of the third wavelength is sampled; andwherein the first and second semiconductor portions of each photodetector of the first array are located at a different depth, relativeto each other and to a surface of each of the photo detectors from thesemiconductor portion of each photo detector of the second array.

In this embodiment, the digital camera may further include a first lensdisposed in and associated with an optical path of the first array ofphoto detectors and a second lens disposed in and associated with anoptical path of the second array of photo detectors wherein the firstlens passes light of the first and second wavelengths and filters lightof the third wavelength. Indeed, the digital camera may include anoptical filter disposed in and associated with an optical path of thefirst array of photo detectors wherein the optical filter passes lightof the first and second wavelengths and filters light of the thirdwavelength. Moreover, the first array of detectors may sample theintensity of light of the first wavelength for a first integration timeand the intensity of light of the second wavelength for a secondintegration time; and the second array of photo detectors may sample theintensity of light of the third wavelength for a third integration time.

The signal processing circuitry of the digital camera may generate afirst image using data which is representative of the intensity of lightsampled by the first array of photo detectors, and a second image usingdata which is representative of the intensity of light sampled by thesecond array of photo detectors. Thereafter, the signal processingcircuitry may generate the composite image using the first image and thesecond image.

The digital camera may further include a memory to store (i) data whichis representative of the intensity of light sampled by the first arrayof photo detectors, and (ii) data which is representative of theintensity of light sampled by the second array of photo detectors. Thememory, the first array of photo detectors, the second array of photodetectors, and the signal processing circuitry may be integrated on orin the same semiconductor substrate.

Further, timing and control logic may be included to provide timing andcontrol information to the signal processing circuitry, the first arrayof photo detectors and/or the second array of photo detectors. Inaddition, communication circuitry (wireless, wired and/or opticalcommunication circuitry) to output data which is representative of thecomposite image. The communication circuitry, memory, the first array ofphoto detectors, the second array of photo detectors, and the signalprocessing circuitry may be integrated on or in the same semiconductorsubstrate.

In any of the embodiments above, the first array of photo detectors mayinclude a first surface area and the second array of photo detectorsincludes a second surface area wherein the first surface area isdifferent from the second surface area. Moreover, the photo detectors ofthe first array may include a first active surface area and the photodetectors of the second array may include a second active surface areawherein the first active surface area is different from the secondactive surface area.

In addition, in any of the embodiments, the first array of photodetectors may include a first surface area and the second array of photodetectors includes a second surface area wherein the first surface areais substantially the same as the second surface area. The photodetectors of the first array may include a first active surface area andthe photo detectors of the second array may include a second activesurface area wherein the first active surface area is different from thesecond active surface area.

A digital camera comprising a plurality of arrays of photo detectors,including a first array of photo detectors to sample an intensity oflight of a first wavelength (which may be associated with a first color)and a second array of photo detectors to sample an intensity of light ofa second wavelength (which may be is associated with a second color).The digital camera further may also include a first lens (which may passlight of the first wavelength onto an image plane of the photo detectorsof the first array and may filter/attenuate light of the secondwavelength) disposed in an optical path of the first array of photodetectors, wherein the first lens includes a predetermined opticalresponse to the light of the first wavelength, and a second lens (whichmay pass light of the second wavelength onto an image plane of the photodetectors of the second array and may filter/attenuate light of thefirst wavelength) disposed in with an optical path of the second arrayof photo detectors wherein the second lens includes a predeterminedoptical response to the light of the second wavelength. In addition, thedigital camera may include signal processing circuitry, coupled to thefirst and second arrays of photo detectors, to generate a compositeimage using (i) data which is representative of the intensity of lightsampled by the first array of photo detectors, and (ii) data which isrepresentative of the intensity of light sampled by the second array ofphoto detectors; wherein the first array of photo detectors, the secondarray of photo detectors, and the signal processing circuitry areintegrated on or in the same semiconductor substrate.

The digital camera may further include a third array of photo detectorsto sample the intensity of light of a third wavelength (which may be isassociated with a third color) and a third lens disposed in with anoptical path of the third array of photo detectors wherein the thirdlens includes a predetermined optical response to the light of the thirdwavelength. As such, the signal processing circuitry is coupled to thethird array of photo detectors and generates a composite image using (i)data which is representative of the intensity of light sampled by thefirst array of photo detectors, (ii) data which is representative of theintensity of light sampled by the second array of photo detectors, and(ii) data which is representative of the intensity of light sampled bythe third array of photo detectors. The first, second and third arraysof photo detectors may be relatively arranged in a triangularconfiguration (for example, an isosceles, obtuse, acute or a righttriangular configuration).

In one embodiment, the first lens filters light of the second and thirdwavelengths, the second lens filters light of the first and thirdwavelengths, and the third lens filters light of the first and secondwavelengths.

In one embodiment, the first array of photo detectors sample theintensity of light of the first wavelength for a first integration timeand the second array of photo detectors sample the intensity of light ofthe second wavelength for a second integration time. Where the digitalcamera includes a third array, the third array of photo detectors maysample the intensity of light of the third wavelength for a thirdintegration time.

The digital camera may further include a housing, wherein the first andsecond lenses, first and second arrays of photo detectors, and thesignal processing circuitry are attached to the housing, and wherein thefirst and second lenses are independently positionable relative to theassociated array of photo detectors.

In some embodiments, the first array of photo detectors sample anintensity of light of the first wavelength (which is associated with afirst color) and an intensity of light of a third wavelength (which isassociated with a third color) and the second array of photo detectorssample an intensity of light of a second wavelength (which is associatedwith a second color). Here, each photo detector of the first array mayinclude a first semiconductor portion at which the intensity of light ofthe first wavelength is sampled and a second semiconductor portion atwhich the intensity of light of the third wavelength is sampled.Further, each photo detector of the second array may include asemiconductor portion at which the intensity of light of the secondwavelength is sampled. In this embodiment, the first and secondsemiconductor portions of each photo detector of the first array arelocated at a different depth, relative to each other and to a surface ofeach of the photo detectors from the semiconductor portion of each photodetector of the second array.

Further, in one or more of these embodiments, the first lens may passlight of the first and third wavelengths and filters light of a secondwavelength. In addition to, or in lieu thereof, an optical filterdisposed in and associated with an optical path of the first array ofphoto detectors wherein the optical filter passes light of the first andthird wavelengths and filters light of the second wavelength.

Moreover, the first array of photo detectors may sample the intensity oflight of the first wavelength for a first integration time and theintensity of light of the third wavelength for a third integration time.The second array of photo detectors sample the intensity of light of thethird wavelength for a second integration time.

The signal processing circuitry of the digital camera may generate afirst image using data which is representative of the intensity of lightsampled by the first array of photo detectors, and a second image usingdata which is representative of the intensity of light sampled by thesecond array of photo detectors. Thereafter, the signal processingcircuitry may generate the composite image using the first image and thesecond image.

The digital camera may further include a memory to store (i) data whichis representative of the intensity of light sampled by the first arrayof photo detectors, and (ii) data which is representative of theintensity of light sampled by the second array of photo detectors. Thememory, the first array of photo detectors, the second array of photodetectors, and the signal processing circuitry may be integrated on orin the same semiconductor substrate.

Further, timing and control logic may be included to provide timing andcontrol information to the signal processing circuitry, the first arrayof photo detectors and/or the second array of photo detectors. Inaddition, communication circuitry (wireless, wired and/or opticalcommunication circuitry) to output data which is representative of thecomposite image. The communication circuitry, memory, the first array ofphoto detectors, the second array of photo detectors, and the signalprocessing circuitry may be integrated on or in the same semiconductorsubstrate.

The signal processing circuitry may include first signal processingcircuitry and second signal processing circuitry wherein the firstsignal processing circuitry is coupled to and associated with the firstarray of photo detectors and second signal processing circuitry iscoupled to and associated with the second array of photo detectors. Inaddition, the signal processing circuitry includes first analog signallogic and second analog signal logic wherein the first analog signallogic is coupled to and associated with the first array of photodetectors and second analog signal logic is coupled to and associatedwith the second array of photo detectors. Moreover, the signalprocessing circuitry may include first black level logic and secondblack level logic wherein the first black level logic is coupled to andassociated with the first array of photo detectors and second blacklevel logic is coupled to and associated with the second array of photodetectors. Notably, the signal processing circuitry includes firstexposure control circuitry and second exposure control circuitry whereinthe first exposure control circuitry is coupled to and associated withthe first array of photo detectors and second exposure control circuitryis coupled to and associated with the second array of photo detectors.

The digital camera may include a frame, wherein the first and secondarrays of photo detectors, the signal processing circuitry, and thefirst and second lenses are fixed to the frame.

In any of the embodiments above, the first array of photo detectors mayinclude a first surface area and the second array of photo detectorsincludes a second surface area wherein the first surface area isdifferent from the second surface area. Moreover, the photo detectors ofthe first array may include a first active surface area and the photodetectors of the second array may include a second active surface areawherein the first active surface area is different from the secondactive surface area.

In addition, in any of the embodiments, the first array of photodetectors may include a first surface area and the second array of photodetectors includes a second surface area wherein the first surface areais substantially the same as the second surface area. The photodetectors of the first array may include a first active surface area andthe photo detectors of the second array may include a second activesurface area wherein the first active surface area is different from thesecond active surface area.

Again, there are many inventions described and illustrated herein. ThisSummary of the Invention is not exhaustive of the scope of the presentinventions. Moreover, this Summary of the Invention is not intended tobe limiting of the invention and should not be interpreted in thatmanner. Thus, while certain embodiments have been described and/oroutlined in this Summary of the Invention, it should be understood thatthe present invention is not limited to such embodiments, descriptionand/or outline. Indeed, many others embodiments, which may be differentfrom and/or similar to, the embodiments presented in this Summary, willbe apparent from the description, illustrations and/or claims, whichfollow.

In addition, although various features, attributes and advantages havebeen described in this Summary of the Invention and/or are apparent inlight thereof, it should be understood that such features, attributesand advantages are not required whether in one, some or all of theembodiments of the present inventions, and indeed, except where statedotherwise, need not be present in any of the aspects and/or embodimentsof the present invention.

Various objects, features and/or advantages of one or more aspectsand/or embodiments of the present invention will become more apparentfrom the following detailed description and the accompanying drawings inwhich like numerals represent like components. It should be understoodhowever, that any such objects, features, and/or advantages are notrequired, and indeed, except where stated otherwise, need not be presentin any of the aspects and/or embodiments of the present invention.

It should be understood that the various aspects and embodiments of thepresent invention that do not appear in the claims that follow arepreserved for presentation in one or more divisional/continuation patentapplications.

BRIEF DESCRIPTION OF DRAWINGS

In the course of the detailed description to follow, reference will bemade to the attached drawings. These drawings show different aspects andembodiments of the present invention and, where appropriate, referencenumerals illustrating like structures, components, materials and/orelements in different figures are labeled similarly. It is understoodthat various combinations of the structures, components, materialsand/or elements, other than those specifically shown, are contemplatedand are within the scope of the present invention.

FIG. 1A illustrates a prior art digital camera, and its primarycomponents;

FIGS. 1B-1D are schematic illustrations of the prior art image capturingelements of the prior art digital camera of FIG. 1A;

FIG. 1E shows the operation of the lens assembly of the prior art cameraof FIG. 1A, in a retracted mode;

FIG. 1F shows the operation of the lens assembly of the prior art cameraof FIG. 1A, in an optical zoom mode;

FIG. 2 illustrates a digital camera, and its primary components,including a digital camera subsystem (DCS) in accordance with oneembodiment of aspects of the invention;

FIGS. 3A-3B are schematics of a digital camera subsystem (DCS);

FIG. 4 illustrates a digital camera subsystem having a three array/lensconfiguration.

FIGS. 5A-5C is a schematic of image capture using the digital camerasubsystem (DCS) of FIGS. 2-3;

FIG. 6A is an alternative digital camera subsystem (DCS) having fourarrays;

FIG. 6B is a flow chart for the alternative digital camera subsystem(DCS) of FIG. 6A;

FIGS. 7A-7C are a schematic of a four-lens system used in the DCS ofFIG. 3;

FIG. 8 is a schematic representation of a digital camera apparatus inaccordance with another embodiment of aspects of the present invention;

FIG. 9A is a schematic exploded representation of an optics portion thatmay be employed in a digital camera apparatus in accordance with oneembodiment of the present invention;

FIGS. 9B-9D are schematic exploded representations of optics portionsthat may be employed in a digital camera apparatus in accordance withfurther embodiments of the present invention;

FIGS. 10A-10H are schematic representations of optics portions that maybe employed in a digital camera apparatus in accordance with furtherembodiments of the present invention;

FIGS. 11A-11B are schematic and side elevational views, respectively, ofa lens used in an optics portion adapted to transmit red light or a redband of light, e.g., for a red camera channel, in accordance withanother embodiment of the present invention;

FIGS. 12A-12B are schematic and side elevational views, respectively, ofa lens used in an optics portion adapted to transmit green light or agreen band of light, e.g., for a green camera channel, in accordancewith another embodiment of the present invention;

FIGS. 13A-13B are schematic and side elevational views, respectively, ofa lens used in an optics portion adapted to transmit blue light or ablue band of light, e.g., for a blue camera channel, in accordance withanother embodiment of the present invention;

FIG. 14 is a schematic view of a lens used in an optics portion adaptedto transmit red light or a red band of light, e.g., for a red camerachannel, in accordance with another embodiment of the present invention;

FIGS. 15A-15F are schematic representations of lenses that may beemployed in a digital camera apparatus in accordance with furtherembodiments of the present invention;

FIG. 16A is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in a digital camera apparatusin accordance with one embodiment of the present invention;

FIG. 16B is a schematic representation of a pixel of the sensor array ofFIG. 16A;

FIG. 16C is a schematic representation of circuit that may be employedin the pixel of FIG. 16B, in accordance with one embodiment of thepresent invention;

FIG. 17A is a schematic representation of a portion of a sensor array inaccordance with another embodiment of the present invention;

FIGS. 17B-17K are schematic cross sectional views of various embodimentsof one or more pixels in accordance with further embodiments of thepresent invention; such pixel embodiments may be implemented in any ofthe embodiments described and/or illustrated herein;

FIG. 17F are explanatory representations of sensor arrays in accordancewith further embodiments of the present invention;

FIGS. 18A-18B depict an image being captured by a portion of a sensorarray, in accordance with one embodiment of the present invention;

FIGS. 19A-19B depict an image being captured by a portion of a sensorarray in accordance with another embodiment of the present invention;

FIGS. 20A-20B are schematic representations of a relative positioningprovided for an optics portion and a respective sensor array inaccordance with further embodiments of the present invention;

FIG. 21 is a schematic representation of a relative positioning that maybe provided for four optics portions and four sensor arrays, inaccordance with one embodiment of the present invention;

FIGS. 22A-22B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with one embodimentof the present invention;

FIGS. 23A-23B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 24A-24B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 25A-25B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 26A-26B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 27A-27B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIG. 28A is a schematic perspective view of a support and opticsportions that may be seated therein, in accordance with one embodimentof the present invention;

FIG. 28B is an enlarged schematic plan view of the support of FIG. 28A;

FIG. 28C is an enlarged schematic cross sectional view of the support ofFIG. 28A, taken along the direction A-A of FIG. 28B;

FIG. 28D is an enlarged exploded schematic cross sectional view of aportion of the support of FIG. 28A, taken along the direction A-A ofFIG. 28B; and a lens that may be seated therein;

FIG. 29A is a schematic cross sectional view of a support and opticsportions seated therein, in accordance with another embodiment of thepresent invention;

FIG. 29B is a schematic cross sectional view of a support and opticsportions seated therein, in accordance with another embodiment of thepresent invention;

FIG. 30A is a schematic perspective view of a support and opticsportions that may be seated therein, in accordance with anotherembodiment of the present invention;

FIG. 30B is a schematic plan view of the support of FIG. 30A;

FIG. 30C is a schematic cross sectional view of the support of FIG. 30A,taken along the direction A-A of FIG. 30B;

FIG. 30D is a schematic cross sectional view of the support of FIG. 30A,taken along the direction A-A of FIG. 30B; and a lens that may be seatedtherein;

FIG. 31A is a schematic perspective view of a support and opticsportions that may be seated therein, in accordance with anotherembodiment of the present invention;

FIG. 31B is a schematic plan view of the support of FIG. 31A;

FIG. 31C is a schematic cross sectional view of the support of FIG. 31A,taken along the direction A-A of FIG. 31B;

FIG. 31D is a schematic cross sectional view of the support of FIG. 31A,taken along the direction A-A of FIG. 31B; and a lens that may be seatedtherein;

FIG. 32 is a schematic cross-sectional view of a digital cameraapparatus and a printed circuit board on which the digital cameraapparatus may be mounted; in accordance with one embodiment of thepresent invention;

FIGS. 33A-33F shows one embodiment for assembling and mounting thedigital camera apparatus of FIG. 32;

FIG. 33G is a schematic perspective view of a digital camera apparatusin accordance with another embodiment of the present invention;

FIGS. 33H-33K are schematic elevational views of mounting and electricalconnector configurations that may be employed in association with adigital camera apparatus in accordance with further embodiments of thepresent invention;

FIG. 34 is a schematic cross section view of a support that may beemployed to support the optics portions of FIGS. 11A-11B, 13A-13B, atleast in part, in accordance with another embodiment of the presentinvention.

FIGS. 35A-35C show one embodiment for assembling three lenslets of anoptics portion in the support.

FIG. 36 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 34 and the optics portionsof FIGS. 11A-11B, 13A-13B, and a printed circuit board on which thedigital camera apparatus may be mounted; in accordance with oneembodiment of the present invention;

FIG. 37 is a schematic cross sectional view of another support that maybe employed to support the optics portions of FIGS. 11A-11B, 13A-13B, atleast in part, in accordance with another embodiment of the presentinvention;

FIG. 38 is a schematic cross sectional view of another support that maybe employed to support the optics portions of FIGS. 11A-11B, 13A-13B, atleast in part, in accordance with another embodiment of the presentinvention;

FIG. 39 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 37 and the optics portionsof FIGS. 11A-11B, 13A-13B, and a printed circuit board on which thedigital camera apparatus may be mounted; in accordance with oneembodiment of the present invention;

FIG. 40 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 38 and the optics portionsof FIGS. 11A-11B, 13A-13B, and a printed circuit board on which thedigital camera apparatus may be mounted; in accordance with oneembodiment of the present invention;

FIGS. 41A-41D, are schematic cross sectional views of seatingconfigurations that may be employed in a digital camera apparatus tosupport the lenses of FIGS. 15A-15D, respectively, at least in part, inaccordance with further embodiments of the present invention;

FIGS. 42-44 are schematic cross sectional views of supports that employthe seating configurations of FIGS. 41B-41D, respectively, and may beemployed to support the lenses shown in FIGS. 15B-15D, respectively, atleast in part, in accordance with further embodiments of the presentinvention;

FIG. 45 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 42 and a printed circuitboard on which the digital camera apparatus may be mounted; inaccordance with one embodiment of the present invention;

FIG. 46 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 43 and a printed circuitboard on which the digital camera apparatus may be mounted; inaccordance with one embodiment of the present invention;

FIG. 47 is a schematic cross-sectional view of a digital cameraapparatus that includes the support of FIG. 44 and a printed circuitboard on which the digital camera apparatus may be mounted; inaccordance with one embodiment of the present invention;

FIG. 48 is a schematic representation of a digital camera apparatus inaccordance with another embodiment of the present invention;

FIG. 49 is a schematic cross-sectional view of a digital cameraapparatus and a printed circuit board of a digital camera on which thedigital camera apparatus may be mounted, in accordance with anotherembodiment of the present invention.

FIGS. 50A-50F shows one embodiment for assembling and mounting thedigital camera apparatus of FIG. 49.

FIG. 51 is a schematic representation of a digital camera apparatus thatincludes a spacer in accordance with another embodiment of the presentinvention.

FIG. 52 is a schematic representation of a digital camera apparatus thatincludes a spacer, in accordance with another embodiment of the presentinvention.

FIG. 53 is a schematic cross-sectional view of a digital cameraapparatus and a printed circuit board of a digital camera on which thedigital camera apparatus may be mounted, in accordance with anotherembodiment of the present invention.

FIGS. 54A-54F shows one such embodiment for assembling and mounting thedigital camera apparatus of FIG. 53.

FIG. 55 is a schematic representation of a digital camera apparatus thatincludes a second device and a spacer, in accordance with anotherembodiment of the present invention;

FIG. 56 is a schematic cross-sectional view of a digital cameraapparatus and a printed circuit board of a digital camera on which thedigital camera apparatus may be mounted, in accordance with anotherembodiment of the present invention.

FIGS. 57A-57F shows one such embodiment for assembling and mounting thedigital camera apparatus of FIG. 56;

FIGS. 58-62 are schematic cross-sectional views of digital cameraapparatus and printed circuit boards of digital cameras on which thedigital camera apparatus may be mounted, in accordance with furtherembodiments of the present invention;

FIGS. 63-67 are schematic cross-sectional views of digital cameraapparatus and printed circuit boards of digital cameras on which thedigital camera apparatus may be mounted, in accordance with furtherembodiments of the present invention.

FIGS. 68-72 are schematic cross-sectional views of digital cameraapparatus and printed circuit boards of digital cameras on which thedigital camera apparatus may be mounted, in accordance with furtherembodiments of the present invention;

FIGS. 73A-73B are schematic elevational and cross sectional views,respectively, of a support in accordance with another embodiment of thepresent invention;

FIG. 74 is a schematic cross sectional view of a support in accordancewith another embodiment of the present invention;

FIG. 75 is a schematic plan view of a support in accordance with anotherembodiment of the present invention;

FIG. 76A is a schematic view of a digital camera apparatus that includesone or more output devices in accordance with another embodiment of thepresent invention;

FIGS. 76B-76C are schematic front and rear elevational viewsrespectively, of a display device that may be employed in the digitalcamera apparatus of FIG. 76A, in accordance with one embodiment of thepresent invention;

FIGS. 76D-76E are schematic views of digital camera apparatus thatinclude one or more output devices in accordance with furtherembodiments of the present invention;

FIG. 77A is a schematic view of a digital camera apparatus that includesone or more input devices in accordance with another embodiment of thepresent invention;

FIGS. 77B-77C are enlarged schematic front and rear perspective viewsrespectively, of an input device that may be employed in the digitalcamera apparatus of FIG. 77A, in accordance with one embodiment of thepresent invention;

FIGS. 77D-77L are schematic views of digital camera apparatus thatinclude one or more output devices in accordance with furtherembodiments of the present invention;

FIGS. 77M-77N are schematic plan and cross sectional views,respectively, of a support in accordance with another embodiment of thepresent invention;

FIG. 78A is a schematic view of a digital camera apparatus that includesone or more illumination devices in accordance with another embodimentof the present invention;

FIGS. 78B-78C are enlarged schematic front and rear perspective viewsrespectively, of an illumination device that may be employed in thedigital camera apparatus of FIG. 78A, in accordance with one embodimentof the present invention;

FIGS. 78D-78L are schematic perspective views of digital cameraapparatus that include one or more illumination devices in accordancewith further embodiments of the present invention;

FIGS. 78M-78N are schematic views of digital camera apparatus thatinclude one or illumination devices in accordance with furtherembodiments of the present invention;

FIGS. 79A-79C are schematic perspective views of digital cameraapparatus that include one or more input devices and one or more outputdevices, in accordance with further embodiments of the presentinvention.

FIGS. 80A-80F are schematic perspective views of digital cameraapparatus that include one or more input devices, one or more displaydevices and one or more illumination devices, in accordance with furtherembodiments of the present invention.

FIG. 81A is a schematic perspective view a digital camera apparatus thatincludes molded plastic packaging in accordance with one embodiment ofthe present invention;

FIGS. 81B-81C are schematic exploded perspective views of the digitalcamera apparatus of FIG. 81A;

FIG. 82 is an enlarged schematic front perspective view of a digitalcamera apparatus in accordance with another embodiment of the presentinvention;

FIGS. 83A-83C are schematic front perspective views of sensor array andprocessor configurations, in accordance with further embodiments of thepresent invention;

FIGS. 83A-83C are schematic front perspective views of sensor arrayconfigurations, in accordance with further embodiments of the presentinvention;

FIGS. 84A-84E are schematic representations of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 85A-85E are schematic representations of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 86A-86E are schematic representations of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 87A-8B are schematic representations of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 88A-88E are schematic representations of a digital cameraapparatus in accordance with another embodiment of the presentinvention;

FIGS. 88A-88E are schematic representation of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 89A-89E are schematic representation of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 90A, 91A-91B, 92A-92B, 93A-93, 94A-94B, 95A-95B and 96A-96B are aschematic plan views and a schematic cross sectional view, respectively,of some embodiments of the image device;

FIG. 90A is a schematic plan view of an image device in accordance withanother embodiment of the present invention;

FIGS. 90A-90B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with one embodimentof the present invention;

FIGS. 91A-91B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 92A-92B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 93A-93B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 94A-94B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 95A-95B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIGS. 96A-96B are a schematic plan view and a schematic cross sectionalview, respectively, of an image device in accordance with anotherembodiment of the present invention;

FIG. 97A is a schematic plan view of a support and optics portions thatmay be seated therein, in accordance with one embodiment of the presentinvention;

FIG. 97B is a schematic cross sectional view of the support of FIG. 97A,taken along the direction A-A of FIG. 97B;

FIG. 97C is an exploded schematic cross sectional view of a portion ofthe support of FIG. 97A and a lens that may be seated therein;

FIGS. 99A-99D are schematic representations of digital camera apparatusin accordance with further embodiments of the present invention;

FIGS. 100A-100D are schematic representations of digital cameraapparatus in accordance with further embodiments of the presentinvention;

FIG. 101A is schematic front perspective view of an image device inaccordance with another embodiment of the present invention;

FIG. 101B is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.101A, in accordance with one embodiment of the present invention;

FIG. 101C is a schematic representation of a pixel of the sensor arrayof FIG. 101B;

FIG. 101D is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.101A, in accordance with one embodiment of the present invention;

FIG. 101E is a schematic representation of a pixel of the sensor arrayof FIG. 101D;

FIG. 101F is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.101A, in accordance with one embodiment of the present invention;

FIG. 101G is a schematic representation of a pixel of the sensor arrayof FIG. 101F;

FIG. 102A is schematic front perspective view of an image device inaccordance with another embodiment of the present invention;

FIG. 102B is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.102A, in accordance with one embodiment of the present invention;

FIG. 102C is a schematic representation of a pixel of the sensor arrayof FIG. 102B;

FIG. 102D is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.102A, in accordance with one embodiment of the present invention;

FIG. 102E is a schematic representation of a pixel of the sensor arrayof FIG. 102D;

FIG. 102F is a schematic representation of a sensor array and circuitsconnected thereto, which may be employed in the image device of FIG.102A, in accordance with one embodiment of the present invention;

FIG. 102G is a schematic representation of a pixel of the sensor arrayof FIG. 102F;

FIGS. 103A-103E are schematic representations of digital cameraapparatus in accordance with further embodiments of the presentinvention;

FIGS. 104A-104E are schematic representations of digital cameraapparatus in accordance with further embodiments of the presentinvention;

FIGS. 105A-105E are schematic representations of digital cameraapparatus in accordance with further embodiments of the presentinvention;

FIGS. 106A-106C are schematic perspective views of a system having aplurality of digital camera apparatus, in accordance with anotherembodiment of the present invention;

FIG. 107A is a schematic perspective view of a system having a pluralityof digital camera apparatus, in accordance with another embodiment ofthe present invention;

FIG. 107B is a schematic elevational view of image devices that may beemployed in the system of FIG. 107A;

FIGS. 108A-108B are schematic representations of digital cameraapparatus in accordance with further embodiments of the presentinvention;

FIGS. 109A-109E are block diagram representations showing configurationsof digital camera apparatus in accordance embodiments of the presentinvention;

FIG. 110A is a block diagram of a processor in accordance with oneembodiment of the present invention;

FIG. 110B is a block diagram of a channel processor that may be employedin the processor of FIG. 110A, in accordance with one embodiment of thepresent invention;

FIG. 110C is a block diagram of an image pipeline that may be employedin the processor of FIG. 110A, in accordance with one embodiment of thepresent invention;

FIG. 110D is a block diagram of a post processor that may be employed inthe processor of FIG. 110A, in accordance with one embodiment of thepresent invention;

FIG. 110E is a block diagram of a system control and other portions of adigital camera apparatus, in accordance with one embodiment of thepresent invention;

FIG. 110F is representation of an instruction format according to oneembodiment of the present invention;

FIG. 111A is a block diagram of a channel processor in accordance withanother embodiment of the present invention;

FIG. 111B is a graphical representation of a neighborhood of pixelvalues.

FIG. 111C shows a flowchart of operations employed in one embodiment ofa double sampler;

FIG. 111D shows a flowchart of operations employed in one embodiment ofa defective pixel identifier;

FIG. 111E is a block diagram of an image pipeline in accordance withanother embodiment of the present invention;

FIG. 111F is a schematic diagram of an image plane integrator, inaccordance with one embodiment of the present invention;

FIG. 111G is an explanatory representation of a multi-phase clock thatmay be employed in the image plane integrator of FIG. 111G;

FIGS. 111H-111J are explanatory views showing representations of imagesgenerated by three camera channels, in accordance with one embodiment ofthe present invention;

FIGS. 111K-111Q are explanatory views showing a representation of aprocess carried out by the automatic image alignment portion for theimages of FIGS. 111H-111J, in accordance with one embodiment of thepresent invention;

FIG. 111R is a schematic block diagram of an automatic exposure control,in accordance with one embodiment of the present invention;

FIG. 111S is a schematic block diagram of a zoom controller, inaccordance with one embodiment of the present invention;

FIGS. 111T-11V are explanatory views of a process carried out by thezoom controller of FIG. 111S, in accordance with one embodiment of thepresent invention;

FIG. 111W is a graphical representation showing an example of theoperation of a gamma correction portion, in accordance with oneembodiment of the present invention

FIG. 111X is a schematic block diagram of a gamma correction portionemployed in accordance with one embodiment of the present invention;

FIG. 111Y is a schematic block diagram of a color correction portion, inaccordance with one embodiment of the present invention;

FIG. 111Z is a schematic block diagram of an edge enhancer/sharpener, inaccordance with one embodiment of the present invention;

FIG. 111AA is a schematic block diagram of a chroma noise reductionportion in accordance with one embodiment of the present invention;

FIG. 111AB is an explanatory view showing a representation of a processcarried out by a white balance portion, in accordance with oneembodiment of the present invention;

FIG. 111AC is a schematic block diagram of a color enhancement portion,in accordance with one embodiment of the present invention;

FIG. 111AD is a schematic block diagram of a scaling portion, inaccordance with one embodiment of the present invention;

FIG. 111AE is an explanatory view, showing a representation ofupscaling, in accordance with one embodiment;

FIG. 111AF is a flowchart of operations that may be employed in thealignment portion, in accordance with another embodiment of the presentinvention;

FIG. 112 is a block diagram of a channel processor in accordance withanother embodiment of the present invention;

FIG. 113 is a block diagram of a channel processor in accordance withanother embodiment of the present invention;

FIG. 114A is a block diagram of an image pipeline in accordance withanother embodiment of the present invention;

FIG. 114B is a block diagram of an image pipeline in accordance withanother embodiment of the present invention;

FIG. 114C is a schematic block diagram of a chroma noise reductionportion in accordance with another embodiment of the present invention

FIGS. 115A-115L are explanatory views showing examples of parallax;

FIG. 115M is an explanatory view showing an image viewed by a firstcamera channel superimposed with an image viewed by a second camerachannel if parallax is eliminated, in accordance with one embodiment ofthe present invention;

FIGS. 115N-115R are explanatory representations showing examples ofdecreasing the parallax;

FIGS. 115S-115V and 115X are explanatory views showing examples ofincreasing the parallax;

FIG. 116 shows a flowchart of operations that may be employed ingenerating an estimate of a distance to an object, or portion thereof,according to one embodiment of the present invention.

FIG. 117 is a schematic block diagram of a portion of a range finder, inaccordance with one embodiment of the present invention;

FIG. 118 is a schematic block diagram of a locator portion of the rangefinder, in accordance with one embodiment of the present invention;

FIGS. 119A-119C are explanatory representations showing examples of 3Dimaging;

FIG. 120 is an explanatory representation of another type of 3D imaging;

FIGS. 121-122 show a flowchart of operations that may be employed in 3Dimaging, according to another embodiment of the present invention;

FIG. 123 is a schematic block diagram of a 3D effect generator inaccordance with one embodiment of the present invention;

FIG. 124 is a schematic block diagram of a 3D effect generator inaccordance with one embodiment of the present invention;

FIG. 125 shows a flowchart of operations that may be employed in imagediscrimination, according to another embodiment of the presentinvention;

FIGS. 126A-126B illustrate a flowchart of operations that may beemployed in image discrimination, according to another embodiment of thepresent invention;

FIG. 127 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 128 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 129 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 130 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 131 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 132 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 133 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 134 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 135 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 136 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention;

FIG. 137 is a schematic block diagram representation of one or moreportions of a digital camera apparatus in accordance with anotherembodiment of the present invention; and

FIG. 138 is one or more aspects/techniques/embodiments for implementingspectral optimization of one or more components of a digital cameraapparatus in accordance with further embodiments of the presentinvention; one or more of the aspects/techniques/embodiments may beimplemented in any of the embodiments described and/or illustratedherein.

DETAILED DESCRIPTION

In FIG. 1A a prior art digital camera 1 generally includes the primaryimage capturing elements of an image sensor 150, a color filter sheet160 and a series of lenses 170 (in a lens assembly). Additionalelectronic components typically include a circuit board 110, aperipheral user interface electronics 120 (here represented as a shutterbutton, but could also include display, settings, controls, etc), powersupply 130, and electronic image storage media 140

The digital camera 1 further includes a housing (including housingportions 173, 174, 175, 176, 177 and 178) and a shutter assembly (notshown), which controls an aperture 180 and passage of light into thedigital camera 1. A mechanical frame 181 is used to hold the variousparts of the lens assembly together. The lens assembly includes thelenses 170 and one or more electromechanical devices 182 to move thelenses 170 along an axis 183. The mechanical frame 181 and the one ormore electromechanical devices 182 may be made up of numerous componentsand/or complex assemblies.

The color filter sheet 160 has an array of color filters arranged in aBayer pattern. A Bayer pattern historically uses filters of red, green,blue and typically a second green (e.g., a 2×2 matrix of colors withalternating red and green in one row and alternating green and blue inthe other row, although other colors may be used) although patterns mayvary depending on the need of the customer. The Bayer pattern isrepeated throughout the color filter array 112 as illustrated in FIGS.1A-1D. This pattern is repeated over the entire array as illustrated.

The image sensor 150 contains a plurality of identical photo detectors(sometimes referred to as “picture elements” or “pixels”) arranged in amatrix. The number of photo detectors is usually in the range ofhundreds of thousands to millions. The lens assembly spans the diagonalof the array.

The color filter array 160 is laid over the image sensor 150 such thateach of the color filters in the color filter sheet 160 is disposedabove a respective one of the photo detectors in the image sensor 150,whereby each photo detector in the image sensor receives a specific bandof visible light (e.g., red, green or blue).

FIGS. 1B-1D illustrate the photon capture process used by the prior artdigital camera 1 in creating a color image. The full spectrum of visiblelight 184 strikes the set of lenses, which essentially passes along thefull spectrum. The full spectrum then strikes the color filters of thecolor filter sheet 160 and each of the individual color filters of thecolor filter sheet 160 passes its specific band of spectrum on to itsspecific pixel. This process is repeated for every pixel. Each pixelprovides a signal indicative of the color intensity received thereby.Signal processing circuitry (not shown) receives alternating colorsignals from the photo detectors, processes them in uniform fashion byintegrating each set of four pixels (red/green/blue/green or variationthereof) into a single full color pixel, and ultimately outputs a colorimage.

FIG. 1E shows the operation of the lens assembly in a retracted mode(sometimes referred to as normal mode or a near focus setting). The lensassembly is shown focused on a distant object (represented as alightning bolt) 186. A representation of the image sensor 150 isincluded for reference purposes. A field of view for the camera 1 isdefined between reference lines 188, 190. The width of the field of viewmay be for example, 50 millimeters (mm). To achieve this field of view188, 190, the one or more electromechanical devices 182 have positionedlenses 170 relatively close together. The lens assembly passes the fieldof view through the lenses 170 and onto the image sensor 150 asindicated by reference lines 192, 194. An image of the object (indicatedat 196) is presented onto the image sensor 150 in the same ratio as thewidth of the actual image 186 relative to the actual field of view 188,190.

FIG. 1F shows the operation of the lens assembly 110 in an optical zoommode (sometimes referred to as a far focus setting). In this mode, theone or more electromechanical devices 182 of the lens assemblyre-position the lens 170 so as to reduce the field of view 188, 190 overthe same image area, thus making the object 186 appear closer (i.e.,larger). One benefit of the lens assembly is that the resolution withthe lens assembly in zoom mode is typically equal to the resolution withthe lens assembly in retracted mode. One drawback, however, is that thelens assembly can be costly and complex. Moreover, providing a lens withzoom capability results in less light sensitivity and thus increases theF-stop of the lens, thereby making the lens less effective in low lightconditions.

Some other drawbacks associated with the traditional digital camera 1are as follows.

Traditional digital cameras, employing one large array on an imagesensor, also employ one lens that must span the entire array. Thatcreates two physical size related issues: 1) a lens that spans a largearray (e.g. 3 Meg pixels) will be physically larger than a lens thatspans a smaller array (e.g., 1 Meg pixels) in both diameter andthickness; and 2) a larger lens/array combination will likely have alonger focal length which will increase the height of the lens.

Also, since the traditional lens must resolve the entire spectrum ofvisible light wavelengths, they are complex, usually with 3-8 separateelements. This also adds to the optical stack height, complexity andcost.

Further, since the traditional lens must pass all bandwidths of color,it must be a clear lens (no color filtering). The needed color filteringpreviously described is accomplished by depositing a sheet of tiny colorfilters beneath the lens and on top of the image sensor. For example, animage sensor with one million pixels will require a sheet of one millionindividual color filters. This color filter array technique is costly(non-standard integrated circuit processing), presents a limiting factorin shrinking the size of the pixels (cross-talk of colors between pixelsin the imaging sensor), plus the color filter array material attenuatesthe in-band photon stream passing through it (i.e., reduces lightsensitivity) since in-band transmission of the color filter arraymaterial is less than 100%.

Further, since the lens must be moved forward and backwards with respectto the image sensor, additional time and power are required. This is anundesirable aspect of digital cameras as it creates long delays incapture response time as well as diminished battery capacity.

One or more of the above drawbacks associated with traditional digitalcameras may be addressed by one or more embodiments of one or moreaspects of the present invention, although this is not required.

FIG. 2 shows an example of a digital camera 2, and components thereof,in accordance with one embodiment of certain aspects of the presentinvention. The digital camera includes a digital camera subsystem 200, acircuit board 110, a peripheral user interface electronics (hererepresented as a shutter button, but could also include display and/orone or more other output devices, setting controls and/or one or moreadditional input devices etc) 120, a power supply 130, and electronicimage storage media 140.

The digital camera of FIG. 2 may further include a housing and a shutterassembly (not shown), which controls an aperture 180 and passage oflight into the digital camera 2.

FIGS. 3A-3B are partially exploded, schematic views of one embodiment ofthe digital camera subsystem 200. In this embodiment, the digital camerasubsystem includes an image sensor 210, a frame 220 (FIGS. 7A-7C) andlenses 230A-230D. The image sensor 210 generally includes asemiconductor integrated circuit or “chip” having several higher orderfeatures including multiple arrays 210A-210D and signal processingcircuitries 212, 214. Each of the arrays 210A-210D captures photons andoutputs electronic signals. The signal processing circuitry 212, incertain embodiments, processes signals for each of the individual arrays210. The signal processing circuitry 214 may combine the output fromsignal processing 212 into output data (usually in the form of arecombined full color image). Each array and the related signalprocessing circuitry may be preferably tailored to address a specificband of visible spectrum.

Each of lenses 230A-230D may be advantageously tailored for therespective wavelength of the respective array. Lenses will generally beabout the same size as the underlying array, and will therefore differfrom one another in size and shape depending upon the dimensions of theunderlying array. Of course, there is no requirement that a given lenscover all, or only, the underlying array. In alternative embodiments alens could cover only a portion of an array, and could extend beyond thearray. Lenses can comprise any suitable material or materials, includingfor example, glass and plastic. Lenses can be doped in any suitablemanner, such as to impart a color filtering, polarization, or otherproperty. Lenses can be rigid or flexible.

The frame 220 (FIGS. 7A-7C) is used to mount the lenses 230A-230D to theimage sensor 210.

In this exemplary embodiment, each lens, array, and signal processingcircuitry constitutes an image generating subsystem for a band ofvisible spectrum (e.g., red, blue, green, etc). These individual imagesare then combined with additional signal processing circuitry within thesemiconductor chip to form a full image for output.

Those skilled in the art will appreciate that although the digitalcamera subsystem 210 is depicted in a four array/lens configuration, thedigital camera subsystem can be employed in a configuration having anymultiple numbers and shapes of arrays/lenses.

FIG. 4 depicts a digital camera subsystem having a three array/lensconfiguration.

In FIGS. 5A-5C, the digital camera subsystem employs the separatearrays, e.g., arrays 210A-210D, on one image sensor to supplant theprior art approach (which employs a Bayer pattern (or variationsthereof), operations across the array (a pixel at a time) and integrateseach set of four pixels (for example, red/green/blue/green or variationthereof) from the array into a single full color pixel). Each of sucharrays focuses on a specific band of visible spectrum. As such, eacharray may be tuned so that it is more efficient in capturing andprocessing the image in that particular color. Individual lenses (230A-D) can be tailored for the array's band of spectrum. Each lens onlyneeds to pass that color (184 A-184D) on to the image sensor. Thetraditional color filter sheet is eliminated. Each array outputs signalsto signal processing circuitry. Signal processing circuitry for each ofthese arrays is also tailored for each of the bands of visible spectrum.In effect, individual images are created for each of these arrays.Following this process, the individual images are combined to form onefull color or black/white image. By tailoring each array and theassociated signal processing circuitry, a higher quality image can begenerated than the image resulting from traditional image sensors oflike pixel count.

FIGS. 6A-6B illustrate some of the many processing operations that canbe advantageously used. As stated above, each array outputs signals tothe signal processing circuitry 212. Within the signal processingcircuitry, each array can be processed separately to tailor theprocessing to the respective bands of spectrum. Several functions occur:

The column logic (212.1 A-D) is the portion of the signal processingcircuitry that reads the signals from the pixels. For example, thecolumn logic 212.1A reads signals from the pixels in array 210A. Columnlogic 212.1B reads signals from the pixels in array 210B. Column logic212.1C reads signals from the pixels in array 210C. Column logic 212.1Dreads signals from the pixels in array 210D.

Since the array is targeting a specific wavelength, wavelengths, band ofwavelength, or band of wavelengths, the column logic may have differentintegration times for each array enhancing dynamic range and/or colorspecificity. Signal processing circuitry complexity for each array canbe substantially reduced since logic may not have to switch betweenextreme color shifts.

The Analog Signal Logic (ASL) (212.2 A-D) for each array may be colorspecific. As such, the ASL processes a single color and therefore can beoptimized for gain, noise, dynamic range, linearity, etc. Due to colorsignal separation, dramatic shifts in the logic and settling time arenot required as the amplifiers and logic do not change on a pixel bypixel (color to color) basis as in traditional Bayer patterned designs.

The black level logic (212.3 A-D) assesses the level of noise within thesignal, and filters it out. With each array focused upon a narrower bandof visible spectrum than traditional image sensors, the black levellogic can be more finely tuned to eliminate noise.

The exposure control (212.4 A-D) measures the overall volume of lightbeing captured by the array and will adjust the capture time for imagequality. Traditional cameras must make this determination on a globalbasis (for all colors). Our invention allows for exposure control tooccur for each array and targeted band of wavelengths differently.

These processed images are then passed to the second group of signalprocessing circuitry 214. First, the image processing logic 214.1integrates the multiple color planes into a single color image. Theimage is adjusted for saturation, sharpness, intensity, hue, artifactremoval, and defective pixel correction. The IPL also provides thealgorithmic auto focus, zoom, windowing, pixel binning and camerafunctions.

The final two operations are to encode the signal into standardprotocols 214.2 such as MPEG, JPEG, etc. before passing to a standardoutput interface 214.3, such as USB.

Although the signal processing circuitries 212, 214 are shown atspecific areas of the image sensor, the signal processing circuitries212, 214 can be placed anywhere on the chip and subdivided in anyfashion. The signal processing circuitries in fact will likely be placedin multiple locations.

As previously stated, the image sensor 210 (FIGS. 3A-3B) generallyincludes a semiconductor chip having several higher order featuresincluding multiple arrays (210A-210 D), and signal processing circuitry212, in which each array and the related signal processing circuitry ispreferably tailored to address a specific band of visible spectrum. Asnoted above, the image sensor array can be configured using any multiplenumbers and shapes of arrays.

The image sensor 210 can be constructed using any suitable technology,including especially silicon and germanium technologies. The pixels canbe formed in any suitable manner, can be sized and dimensioned asdesired, and can be distributed in any desired pattern. Pixels that aredistributed without any regular pattern could even be used.

Any range of visible spectrum can be applied to each array depending onthe specific interest of the customer. Further, an infrared array couldalso be employed as one of the array/lens combinations giving low lightcapabilities to the sensor.

As previously described, arrays 210A-D may be of any size or shape. FIG.3 shows the arrays as individual, discrete sections of the image sensor.These arrays may also be touching. There may also be one large arrayconfigured such that the array is subdivided into sections whereby eachsection is focused upon one band of spectrum, creating the same effectas separate arrays on the same chip.

Although the well depth of the photo detectors (for example, an area orportion of the photo detector that captures, collects, is responsive to,detects and/or senses for example, the intensity illumination ofincident light; in some embodiments, the well depth is the distance fromthe surface of the photo detector to a doped region—see, for example,FIGS. 17B-E) across each individual array (designated 210A-D) may be thesame, the well depth of any given array may be different from that ofone or more or all of other arrays of the sensor subsystem. Selection ofan appropriate well depth could depend on many factors, including mostlikely the targeted band of visible spectrum. Since each entire array islikely to be targeted at one band of visible spectrum (e.g., red) thewell depth can be designed to capture that wavelength and ignore others(e.g., blue, green).

Doping of the semiconductor material in the color specific arrays canfurther be used to enhance the selectivity of the photon absorption forcolor specific wavelengths.

In FIGS. 7A-7C, frame 220 is a thin plate bored to carry the individuallenses (represented by 230A, 230C) over each array. Lenses may be fixedto the frame in a wide range of manners: adhesive, press fit, electronicbonding, etc. The mounting holes may have a small “seat” at the base tocontrol the depth of the lens position. The depth may be different foreach lens and is a result of the specific focal length for theparticular lens tailored for each array.

The frames shown in FIGS. 7A-7C are solid devices that offer a widerange of options for manufacturing, material, mounting, size, and shape.Of course, other suitable frames can be readily designed, all of whichfall within the inventive scope.

Although the Figures show individual lenses per array, assembled into aframe, the lenses could be manufactured such that the lenses per imagesensor come as one mold or component. Further, this one-bodyconstruction could also act as the frame for mounting to the imagesensor.

The lens and frame concept can be applied to traditional image sensors(without the traditional color filter sheet) to gain physical size, costand performance advantages.

As shown in FIGS. 7A-7C, the digital camera subsystem can have multipleseparate arrays on a single image sensor, each with its own lens(represented by 230A, 230C). The simple geometry of smaller, multiplearrays allows for a smaller lens (diameter, thickness and focal length),which allows for reduced stack height in the digital camera.

Each array can advantageously be focused on one band of visible and/ordetectable spectrum. Among other things, each lens may be tuned forpassage of that one specific band of wavelength. Since each lens wouldtherefore not need to pass the entire light spectrum, the number ofelements may be reduced, for example, to one or two.

Further, due to the focused bandwidth for each lens, each of the lensesmay be dyed during the manufacturing process for its respectivebandwidth (e.g., red for the array targeting the red band of visiblespectrum). Alternatively, a single color filter may be applied acrosseach lens. This process eliminates the traditional color filters (thesheet of individual pixel filters) thereby reducing cost, improvingsignal strength and eliminating the pixel reduction barrier.

Elimination of the color filter sheet allows for reductions in thephysical size of the pixel for further size reductions of the overallDCS assembly.

Although FIGS. 2, 3 A-3B and 5A-5C illustrates a 4 array/lens structure,and FIG. 4 depicts a three array/lens configuration, any multiple numberof arrays/lenses as well as various combinations thereof is possible.

The above-described devices can include any suitable number ofcombinations, from as few as 2 arrays/lenses or in a broader array.Examples include:

2 arrays/lenses: red/green and blue

2 arrays/lenses: red and blue/green

3 arrays/lenses: red, green, blue

4 arrays/lenses: red, blue, green, emerald (for color enhancement)

4 arrays/lenses: red, blue, green, infrared (for low light conditions)

8 arrays/lenses: double the above configurations for additional pixelcount and image quality.

Although FIG. 2 reflects a digital still camera, the camera is intendedto be emblematic of a generic appliance containing the digital camerasubsystem. Thus, FIG. 2 should be interpreted as being emblematic ofstill and video cameras, cell phones, other personal communicationsdevices, surveillance equipment, automotive applications, computers,manufacturing and inspection devices, toys, plus a wide range of otherand continuously expanding applications. Of course these alternativeinterpretations of the FIG. may or may not include the specificcomponents as depicted in FIG. 2. For example, the circuit board may notbe unique to the camera function but rather the digital camera subsystemmay be an add-on to an existing circuit board, such as in a cell phone.

Thus, it should be understood that any or all of the methods and/orapparatus disclosed herein may be employed in any type of apparatus orprocess including, but not limited to still and video cameras, cellphones, other personal communications devices, surveillance equipment,automotive applications, computers, manufacturing and inspectiondevices, toys, plus a wide range of other and continuously expandingapplications.

As used herein, the following terms are interpreted as described below,unless the context requires a different interpretation.

“Array” means a group of photo detectors, also know as pixels, whichoperate in concert to create one image. The array captures the photonsand converts the data to an electronic signal. The array outputs thisraw data to signal processing circuitry that generates the image sensorimage output.

“Digital Camera” means a single assembly that receives photons, convertsthem to electrical signals on a semiconductor device (“image sensor”),and processes those signals into an output that yields a photographicimage. The digital camera would included any necessary lenses, imagesensor, shutter, flash, signal processing circuitry, memory device, userinterface features, power supply and any mechanical structure (e.g.circuit board, housing, etc) to house these components. A digital cameramay be a stand-alone product or may be imbedded in other appliances,such as cell phones, computers or the myriad of other imaging platformsnow available or may be created in the future, such as those that becomefeasible as a result of this invention.

“Digital Camera Subsystem” (DCS) means a single assembly that receivesphotons, converts them to electrical signals on a semiconductor device(“image sensor”) and processes those signals into an output that yieldsa photographic image. At a minimum, the Digital Camera Subsystem wouldinclude any necessary lenses, image sensor, signal processing circuitry,shutter, flash and any frame to hold the components as may be required.The power supply, memory devices and any mechanical structure are notnecessarily included.

“Electronic media” means that images are captured, processed and storedelectronically as opposed to the use of film.

“Frame” or “thin plate” means the component of the DCS that is used tohold the lenses and mount to the image sensor.

“Image sensor” means the semiconductor device that includes the photondetectors (“pixels”), processing circuitry and output channels. Theinputs are the photons and the output is the image data.

“Lens” means a single lens or series of stacked lenses (a column oneabove the other) that shape light rays above an individual array. Whenmultiple stacks of lenses are employed over different arrays, they arecalled “lenses.”

“Package” means a case or frame that an image sensor (or anysemiconductor chip) is mounted in or on, which protects the imager andprovides a hermetic seal. “Packageless” refers to those semiconductorchips that can be mounted directly to a circuit board without need of apackage.

The terms “Photo-detector” and “pixels” mean an electronic device thatsenses and captures photons and converts them to electronic signals.These extremely small devices are used in large quantities (hundreds ofthousands to millions) in a matrix to capture an image much like film.

“Semiconductor Chip” means a discrete electronic device fabricated on asilicon or similar substrate, which is commonly used in virtually allelectronic equipment.

“Signal Processing Circuitry” means the hardware and software within theimage sensor that translates the photon input information intoelectronic signals and ultimately into an image output signal.

The inventive subject matter can provide numerous benefits in specificapplications. For example, traditional color filters are limited intheir temperature range, which limits end user manufacturingflexibility. Wave soldering processes, low cost, mass productionsoldering processes, cannot be used due to the color filters'temperature limitations. At least some embodiments of the inventivesubject matter do not have that limitation. Indeed, one, some or all ofthe embodiments described and illustrated herein need not employ wavesoldering processes or other low cost, mass production solderingprocesses.

In addition, once the imager sensor, frame, and lenses are assembled,the assembly can be a hermetically sealed device. The device does notneed a “package” and as such, if desired, can be mounted directly to acircuit board which saves parts and manufacturing costs.

Because multiple images are created from separate locations (albeit asmall distance between the arrays on the same image sensor), parallax iscreated, which can be eliminated in the signal processing circuitry orutilized/enhanced for numerous purposes, including for example, tomeasure distance to the object, and to provide a 3-D effect.

Although each array and the related signal processing circuitry ispreferably tailored to address a specific band of visible spectrum andeach lens may be tuned for passage of that one specific band ofwavelength, it should be clear that there is no requirement that eachsuch array and the related signal processing circuitry be tailored toaddress a specific band of the visible spectrum. Nor is there anyrequirement that each lens be tuned for passage of a specific band ofwavelength or that each of the arrays be located on the samesemiconductor device. Indeed, the embodiments described and illustratedherein, including the specific components thereof, need not employwavelength specific features. For example, the arrays and/or signalprocessing circuitry need not be tailored to address a specificwavelength or band of wavelengths.

Notably, in certain embodiments, certain components thereof may betailored to address a specific wavelength or band of wavelengths whileother components of the embodiment are not tailored to address aspecific wavelength or band of wavelengths. For example, the lensesand/or arrays may be tailored to address a specific wavelength or bandof wavelengths and the associated signal processing circuitry is nottailored to address a specific wavelength or band of wavelengths.Moreover, in other embodiments, one or more lenses (of the same ordifferent optical channels) may be tailored to address a specificwavelength or band of wavelengths and the associated array and signalprocessing circuitry is not tailored to address a specific wavelength orband of wavelengths. All such permutations and combinations are intendedto come within the scope of the present inventions. For the sake ofbrevity, all such permutations and combinations are not discussed indetail herein.

In addition, although a digital camera subsystem includes any necessarylenses, image sensor, signal processing circuitry, shutter, flash andany frame to hold the components as may be required, some digital camerasubsystems may not have any requirement for one or more of such. Forexample, some digital camera systems may not require a shutter, a flashand/or a frame to hold the components. Further, some of the digitalcamera subsystems may not require an image sensor that includes each ofthe detectors, the processing circuitry and output channels. Forexample, in some embodiments, one or more of the detectors (or portionsthereof), one or more portions of the processing circuitry and/or one ormore portions of the output channels may be included in separate devicesand/or disposed in separate locations. All such permutations andcombinations are intended to come within the scope of the presentinventions. For the sake of brevity, all such permutations andcombinations are not discussed in detail herein.

FIG. 8 is a schematic exploded perspective view of a digital cameraapparatus 300 in accordance with another embodiment of the presentinvention. The digital camera apparatus 300 includes one or more sensorarrays, e.g., four sensor arrays 310A-310D, one or more optics portions,e.g., four optics portions 330A-330D, and a processor 340. Each of theone or more optics portions, e.g., optics portions 330A-330D, mayinclude, for example, but is not limited to, a lens, and may beassociated with a respective one of the one or more sensor arrays, e.g.,sensor arrays 310A-310D. In some embodiments, a support 320 (see forexample, FIGS. 28A-28D), for example, but not limited to, a frame, isprovided to support the one or more optics portions, e.g., opticsportions 330A-330D, at least in part. Each sensor array and therespective optics portion may define a camera channel. For example, acamera channel 350A may be defined by the optics portion 330A and thesensor array 310A. Camera channel 350B may be defined by the opticsportion 330B and the sensor array 310B. Camera channel 350C may bedefined by optics portion 330C and the sensor array 310C. Camera channel350D may be defined by optics portion 330D and a sensor array 310D. Theoptics portions of the one or more camera channels are collectivelyreferred to herein as an optics subsystem. The sensor arrays of the oneor more camera channels are collectively referred to herein as a sensorsubsystem. The two or more sensor arrays may be integrated in ordisposed on a common substrate, referred to hereinafter as an imagedevice, on separate substrates, or any combination thereof (for example,where the system includes three or more sensor arrays, two or moresensor arrays may be integrated in a first substrate and one or moreother sensor arrays may be integrated in or disposed on a secondsubstrate).

In that regard, with continued reference to FIG. 8, the one or moresensor arrays, e.g., sensor arrays 310A-310D, may or may not be disposedon a common substrate with one another. For example, in some embodimentstwo or more of the sensor arrays are disposed on a common substrate. Insome embodiments, however, one or more of the sensor arrays is notdisposed on the same substrate as one or more of the other sensorarrays.

The one or more camera channels may or may not be identical to oneanother. For example, in some embodiments, the camera channels areidentical to one another. In other embodiments, one or more of thecamera channels are different, in one or more respects, from one or moreof the other camera channels. In some of the latter embodiments, eachcamera channel may be used to detect a different color (or band ofcolors) and/or band of light than that detected by the other camerachannels.

In some embodiments, one of the camera channels, e.g., camera channel350A, detects red light, one of the camera channels, e.g., camerachannel 350B, detects green light, one of the camera channels, e.g.,camera channel 350C, detects blue light. In some of such embodiments,one of the camera channels, e.g., camera channel 350D, detects infraredlight, cyan light, or emerald light. In some other embodiments, one ofthe camera channels, e.g., camera channel 350A, detects cyan light, oneof the camera channels, e.g., camera channel 350B, detects yellow light,one of the camera channels, e.g., camera channel 350C, detects magentalight and one of the camera channels, e.g., camera channel 350D, detectsclear light (black and white). Any other wavelength or band ofwavelengths (whether visible or invisible) combinations can also beused.

The processor 340 is connected to the one or more sensor arrays, e.g.,sensor arrays 310A-310D, via one or more communication links, e.g.,communication links 370A-370D, respectively. A communication link may beany kind of communication link including but not limited to, forexample, wired (e.g., conductors, fiber optic cables) or wireless (e.g.,acoustic links, electromagnetic links or any combination thereofincluding but not limited to microwave links, satellite links, infraredlinks), and combinations thereof, each of which may be public orprivate, dedicated and/or shared (e.g., a network). A communication linkmay employ for example circuit switching or packet switching orcombinations thereof. Other examples of communication links includededicated point-to-point systems, wired networks, and cellular telephonesystems. A communication link may employ any protocol or combination ofprotocols including but not limited to the Internet Protocol.

The communication link may transmit any type of information. Theinformation may have any form, including, for example, but not limitedto, analog and/or digital (a sequence of binary values, i.e. a bitstring). The information may or may not be divided into blocks. Ifdivided into blocks, the amount of information in a block may bepredetermined or determined dynamically, and/or may be fixed (e.g.,uniform) or variable.

As will be further described hereinafter, the processor may include oneor more channel processors, each which is coupled to a respective one(or more) of the camera channels and generates an image based at leastin part on the signal(s) received from the respective camera channel,although this is not required. In some embodiments, one or more of thechannel processors are tailored to its respective camera channel, forexample, as described herein. For example, where one of the camerachannels is dedicated to a specific wavelength or color (or band ofwavelengths or colors), the respective channel processor may be adaptedor tailored to such wavelength or color (or band of wavelengths orcolors). For example, the gain, noise reduction, dynamic range,linearity and/or any other characteristic of the processor, orcombinations of such characteristics, may be adapted to improve and/oroptimize the processor to such wavelength or color (or band ofwavelengths or colors). Tailoring the channel processing to therespective camera channel may facilitate generate an image of a qualitythat is higher than the quality of images resulting from traditionalimage sensors of like pixel count. In addition, providing each camerachannel with a dedicated channel processor may help to reduce orsimplify the amount of logic in the channel processors as the channelprocessor may not need to accommodate extreme shifts in color orwavelength, e.g., from a color (or band of colors) or wavelength (orband of wavelengths) at one extreme to a color (or band of colors) orwavelength (or band of wavelengths) at another extreme.

In operation, an optics portion of a camera channel receives light fromwithin a field of view and transmits one or more portions of such light,e.g., in the form of an image at an image plane. The sensor arrayreceives one or more portions of the light transmitted by the opticsportion and provides one or more output signals indicative thereof. Theone or more output signals from the sensor array are supplied to theprocessor. In some embodiments, the processor generates one or moreoutput signals based, at least in part, on the one or more signals fromthe sensor array. For example, in some embodiments, each of the camerachannels is dedicated to a different color (or band of colors) orwavelength (or band of wavelengths) than the other camera channels andthe processor generates a separate image for each of such camerachannels. In some other embodiments, the processor may generate acombined image based, at least in part, on the images from two or moreof such camera channels. For example, in some embodiments, each of thecamera channels is dedicated to a different color (or band of colors) orwavelength (or band of wavelengths) than the other camera channels andthe processor combines the images from the two or more camera channelsto provide a partial or full color image.

Although the processor 340 is shown separate from the one or more sensorarrays, e.g., sensor arrays 310A-310D, the processor 340, or portionsthereof, may have any configuration and may be disposed in one or morelocations. In some embodiments, one, some or all portions of theprocessor 340 are integrated in or disposed on the same substrate orsubstrates as one or more of the one or more of the sensor arrays, e.g.,sensor arrays 310A-310D. In some embodiments one, some or all portionsof the processor are disposed on one or more substrates that areseparate from (and possibly remote from) one or more substrates on whichone or more of the one or more sensor arrays, e.g., sensor arrays310A-310D, may be disposed. For example, certain operations of theprocessor may be distributed to or performed by circuitry that isintegrated in or disposed on the same substrate or substrates as one ormore of the one or more of the sensor arrays and certain operations ofthe processor are distributed to or performed by circuitry that isintegrated in or disposed on one or more substrates that are differentfrom (whether such one or more different substrates are physicallylocated within the camera or not) the substrates the one or more of thesensor arrays are integrated in or disposed on.

The digital camera apparatus 300 may or may not include a shutter, aflash and/or a frame to hold the components together.

FIG. 9A is a schematic exploded representation of one embodiment of anoptics portion, e.g., optics portions 330A. In this embodiment, theoptics portion 330A includes one or more lenses, e.g., a complexaspherical lens module 380, one or more color coatings, e.g., a colorcoating 382, one or more masks, e.g., an auto focus mask 384, and one ormore IR coatings, e.g., an IR coating 386.

Lenses can comprise any suitable material or materials, including forexample, glass and plastic. Lenses can be doped in any suitable manner,such as to impart a color filtering, polarization, or other property.Lenses can be rigid or flexible. In this regard, some embodiments employa lens (or lenses) having a dye coating, a dye diffused in an opticalmedium (e.g., a lens or lenses), a substantially uniform color filterand/or any other filtering technique through which light passes to theunderlying array.

The color coating 382 helps the optics portion filter (i.e.,substantially attenuate) one or more wavelengths or bands ofwavelengths. The auto focus mask 384 may define one or more interferencepatterns that help the digital camera apparatus perform one or more autofocus functions. The IR coating 386 helps the optics portion 370A filtera wavelength or band of wavelength in the IR portion of the spectrum.

The one or more color coatings, e.g., color coating 382, one or moremasks, e.g., mask 384, and one or more IR coatings, e.g., IR coating 386may have any size, shape and/or configuration. In some embodiments, oneor more of the one or more color coatings, e.g., the color coating 382,are disposed at the top of the optics portion (see, for example, FIG.9B). Some embodiments of the optics portion (and/or components thereof)may or may not include the one or more color coatings, one or more masksand one or more IR coatings and may or may not include features inaddition thereto or in place thereof. In some embodiments, for example,one or more of the one or more color coatings, e.g., the color coating382, are replaced by one or more filters 388 disposed in the opticsportion, e.g., disposed below the lens (see, for example, FIG. 9C). Inother embodiments, one or more of the color coatings are replaced by oneor more dyes diffused in the lens (see, for example, FIG. 9D).

The one or more optics portions, e.g., optics portions 330A-330D, may ormay not be identical to one another. In some embodiments, for example,the optics portions are identical to one another. In some otherembodiments, one or more of the optics portions are different, in one ormore respects, from one or more of the other optics portions. Forexample, in some embodiments, one or more of the characteristics (forexample, but not limited to, its type of element(s), size, response,and/or performance) of one or more of the optics portions is tailored tothe respective sensor array and/or to help achieve a desired result. Forexample, if a particular camera channel is dedicated to a particularcolor (or band of colors) or wavelength (or band of wavelengths) thenthe optics portion for that camera channel may be adapted to transmitonly that particular color (or band of colors) or wavelength (or band ofwavelengths) to the sensor array of the particular camera channel and/orto filter out one or more other colors or wavelengths. In some of suchembodiments, the design of an optical portion is optimized for therespective wavelength or bands of wavelengths to which the respectivecamera channel is dedicated. It should be understood, however, that anyother configurations may also be employed. Each of the one or moreoptics portions may have any configuration.

In some embodiments, each of the optics portions, e.g., optics portions330A-330D, comprises a single lens element or a stack of lens elements(or lenslets), although, as stated above, the present invention is notlimited to such. For example, in some embodiments, a single lenselement, multiple lens elements and/or compound lenses, with or withoutone or more filters, prisms and/or masks are employed.

An optical portion can also contain other optical features that aredesired for digital camera functionality and/or performance. This can bethings such as electronically tunable filters, polarizers, wavefrontcoding, spatial filters (masks), and other features not yet anticipated.Some of the new features (in addition to the lenses) can be electricallyoperated (such as a tunable filter) or be moved mechanically with MEMsmechanisms.

Referring to FIG. 10A-10F, an optics portion, such as for example,optics portion 330A, may include, for example, any number of lenselements, optical coating wavelength filters, optical polarizers and/orcombination thereof. Other optical elements may be included in theoptical stack to create desired optical features. FIG. 10A is aschematic representation of one embodiment of optics portion 330A inwhich the optics portion 330A comprises a single lens element 390. FIG.10B is a schematic representation of another embodiment of the opticsportion 330A in which the optics portion 330A includes two or more lenselements, e.g., lens elements 392A, 392B. The portions of an opticsportion may be separate from one another, integral with one another,and/or any combination thereof. Thus, for example, the two lens elements392A, 392B represented in FIG. 10B may be separate from one another orintegral with one another.

FIGS. 10C-10F show schematic representations of example embodiments ofoptics portion 330A in which the optics portion 330A has one or morelens elements, e.g., lens elements 394A, 394B, and one or more filters,e.g., filter 394C. The one or more lens elements and desired opticalfeatures and/or optical elements may be separate from one another,integral with one another, and/or any combination thereof. Moreover, theone or more lens elements features and/or elements may be disposed inany configuration and/or sequence, for example, a lens-filter sequence(see for example FIG. 10C), lens-coding sequence (see for example FIG.10D), a lens-polarizer sequence (see for example FIG. 10E), alens-filter-coding-polarizer sequence (see for example FIG. 10F) andcombinations and/or variations thereof.

In some embodiments, the filter 394C shown in FIG. 10C is a color filterthat is made within the lenses, deposited on a lens surface or in theoptical system as a separate layer on a support structure. The filtermay be a single band pass or multiple bandpass filter. The coding 396 C(FIG. 10D) may be applied or formed on a lens and/or provided as aseparate optical element. In some embodiments, the coding 396 C is usedto modify the optical wavefront to allow improved imaging capabilitywith additional post image processing. The optical polarizer 400E (FIG.10E), may be of any type to improve image quality such as glarereduction. The polarizer 400E may be applied or formed on one or moreoptical surfaces and/or provided as a dedicated optical element.

FIGS. 10G-10H are schematic representations of optics portions inaccordance with further embodiments of the present invention.

As stated above, the portions of an optics portion may be separate fromone another, integral with one another and/or any combination thereof.If the portions are separate, they may be spaced apart from one another,in contact with one another or any combination thereof. For example, twoor more separate lens elements may be spaced apart from one another, incontact with one another, or any combination thereof. Thus, someembodiments of the optics portion shown in FIG. 10G may be implementedwith the lens elements 402A-402C spaced apart from one another, as isschematically represented in FIG. 10I, or with two or more of the lenselements 402A-402C in contact with one another, as is schematicallyrepresented in FIG. 10I. Further, a filter, e.g., 402D, may beimplemented, for example, as a separate element 402D, as isschematically represented in FIG. 10G, or as a coating 402D disposed onthe surface of a lens, for example, as schematically represented in FIG.10J. The coating may have any suitable thickness and may be, forexample, relatively thin compared to the thickness of a lens, as isschematically represented in FIG. 10K. Similarly, some embodiments ofthe optics portion shown in FIG. 10H may be implemented with the lenselements 404A-404D spaced apart from one another, as is schematicallyrepresented in FIG. 10H, or with two or more of the lens elements404A-404D in contact with one another, as is schematically representedin FIG. 10L. The filter, e.g., filter 404E, may be implemented, forexample, as a separate element 404E, as is schematically represented inFIG. 10H, or as a coating 404E disposed on the surface of a lens, forexample, as schematically represented in FIG. 10M. The coating may haveany suitable thickness and may be, for example, relatively thin comparedto the thickness of a lens, as is schematically represented in FIG. 10N.

It should be understood that such techniques may be employed incombination with any of the embodiments disclosed herein, however, forpurposes of brevity, such embodiments may or may not be individuallyshown and/or discussed herein.

In addition, as with each of the embodiments disclosed herein, it shouldbe understood that any of the embodiments of FIGS. 10A-10N may beemployed in combination with any other embodiments, or portion thereof,disclosed herein. Thus, the embodiments of the optics portions shown inFIGS. 10G-10N may further include a coding and/or a polarizer.

One or more of the camera channels, e.g., 350A-350D, may employ anoptical portion that transmits a narrower band of wavelengths (ascompared to broadband), for example, R, G or B, which in someembodiments, may help to simplify the optical design. For example, insome embodiments, image sharpness and focus is easier to achieve with anoptics portion having a narrow color band than with a traditionaldigital camera that uses a single optical assembly and a Bayer colorfilter array. In some embodiments, the use of multiple camera channelsto detect different bands of colors allows a reduction in the number ofoptical elements in each camera channel. Additional optical approachessuch as diffractive and aspherical surfaces may result in furtheroptical element reduction. In addition, in some embodiments, the use ofoptical portions that transmits a narrower band of wavelengths allowsthe use of color filters that can be applied directly in the opticalmaterial or as coatings. In some embodiments, the optical transmissionin each band is greater than that traditionally provided by the colorfilters utilized with on-chip color filter arrays. In addition, thetransmitted light does not display the pixel to pixel variation that isobserved in color filter array approaches. Further, in some embodiments,the use of multiple optics and corresponding sensor arrays helps tosimplify the optical design and number of elements because the chromaticaberration is much less in a narrower wavelength band as compared tobroadband optics.

In some embodiments, each optical portion transmits a single color orband of colors, multiple colors or bands of colors, or broadband. Insome embodiments, one or more polarizers that polarize the light, whichmay enhance image quality.

In certain embodiments, e.g., if an optical portion transmits multiplebands of colors or broadband, a color filter array (e.g., a color filterarray with a Bayer pattern) may be disposed between the lens and thesensor array and/or the camera channel may employ a sensor array capableof separating the colors or bands of colors.

In some embodiments, an optical portion may itself have the capabilityto provide color separation, for example, similar to that provided by acolor filter array (e.g., a Bayer pattern or variation thereof).

In certain embodiments, a wide range of optics material choices areavailable for the optical portions, including, for example, but notlimited to, molded glasses and plastics.

In some embodiments, one or more photochromic (or photochromatic)materials are employed in one or more of the optical portions. The oneor more materials may be incorporated into an optical lens element or asanother feature in the optical path, for example, above one or more ofthe sensor arrays. In some embodiments, photochromatic materials may beincorporated into a cover glass at the camera entrance (common aperture)to all optics (common to all camera channels), or put into the lenses ofone or more camera channels, or into one or more of the other opticalfeatures included into the optical path of an optics portion over anysensor array.

Some embodiments employ an optics design having a single lens element.Some other embodiments employ a lens having multiple lens elements(e.g., two or more elements). Lenses with multiple lens elements may beused, for example, to help provide better optical performance over abroad wavelength band (such as conventional digital imagers with colorfilter arrays on the sensor arrays). For example, some multi-elementlens assemblies use a combination of single elements to help minimizethe overall aberrations. Because some lens elements have positiveaberrations and others have negative aberrations, the overallaberrations can be reduced. The lens elements may be made of differentmaterials, may have different shapes and/or may define different surfacecurvatures. In this way, a predetermined response may be obtained. Theprocess of determining a suitable and/or optimal lens configuration istypically performed by a lens designer with the aid of appropriatecomputer software.

Some embodiments employ an optics portion having three lens elements orlenslets. The three lenslets may be arranged in a stack of anyconfiguration and spaced apart from one another, wherein each of thelenslets defines two surface contours such that the stack collectivelydefines six surface curvatures and two spaces (between the lenslets). Insome embodiments, a lens with three lenslets provides sufficient degreesof freedom to allow the designer to correct all third order aberrationsand two chromatic aberrations as well as to provide the lens with aneffective focal length, although this is not a requirement for everyembodiment nor is it a requirement for embodiments having three lensletsarranged in a stack.

In that regard, FIGS. 11A-11B are schematic and side elevational views,respectively, of a lens 410 used in an optics portion adapted totransmit red light or a red band of light, e.g., for a red camerachannel, in accordance with another embodiment of the present invention.In this embodiment, the lens 410 includes three lenslets, i.e., a firstlenslet 412, a second lenslet 414 and a third lenslet 416, arranged in astack 418. The lens 410 receives light from within a field of view andtransmits and/or shapes at least a portion of such light to produce animage in an image area at an image plane 419. More particularly, thefirst lenslet 412 receives light from within a field of view andtransmits and/or shapes at least a portion of such light. The secondlenslet 414 receives at least a portion of the light transmitted and/orshaped by the first lenslet and transmits and/or shapes a portion ofsuch light. The third lenslet 416 receives at least a portion of thelight transmitted and/or shaped by the second lenslet and transmitsand/or shapes a portion of such light to produce the image in the imagearea at the image plane 419.

FIGS. 12A-12B are schematic and side elevational views, respectively, ofa lens 420 used in an optics portion adapted to transmit green light ora green band of light, e.g., for a green camera channel, in accordancewith another embodiment of the present invention. In this embodiment,the lens 420 includes three lenslets, i.e., a first lenslet 422, asecond lenslet 424 and a third lenslet 426, arranged in a stack 428. Thestack 428 receives light from within a field of view and transmitsand/or shapes at least a portion of such light to produce an image in animage area at an image plane 429. More particularly, the first lenslet422 receives light from within a field of view and transmits and/orshapes at least a portion of such light. The second lenslet 424 receivesat least a portion of the light transmitted and/or shaped by the firstlenslet and transmits and/or shapes a portion of such light. The thirdlenslet 426 receives at least a portion of the light transmitted and/orshaped by the second lenslet and transmits and/or shapes a portion ofsuch light to produce the image in the image area at the image plane429.

FIGS. 13A-13B are schematic and side elevational views, respectively, ofa lens 430 used in an optics portion adapted to transmit blue light or ablue band of light, e.g., for a blue camera channel, in accordance withanother embodiment of the present invention. In this embodiment, thelens 430 includes three lenslets, i.e., a first lenslet 432, a secondlenslet 434 and a third lenslet 436, arranged in a stack 438. The lens430 receives light from within a field of view and transmits and/orshapes at least a portion of such light to produce an image in an imagearea at an image plane 439. More particularly, the first lenslet 432receives light from within the field of view and transmits and/or shapesat least a portion of such light. The second lenslet 434 receives atleast a portion of the light transmitted and/or shaped by the firstlenslet and transmits and/or shapes a portion of such light. The thirdlenslet 436 receives at least a portion of the light transmitted and/orshaped by the second lenslet and transmits and/or shapes a portion ofsuch light to produce the image in the image area at the image plane439.

FIG. 14 is a schematic view of a lens 440 used in an optics portionadapted to transmit red light or a red band of light, e.g., for a redcamera channel, in accordance with another embodiment of the presentinvention. The lens 440 in this embodiment may be characterized as 60degree, full field of view. In this embodiment, the lens 440 includesthree lenslets, i.e., a first lenslet 442, a second lenslet 444, and athird lenslet 446, arranged in a stack 448. The lens 440 receives lightfrom within a field of view and transmits and/or shapes at least aportion of such light to produce an image in an image area at an imageplane 449. More particularly, the first lenslet 442 receives light fromwithin a field of view and transmits and/or shapes at least a portion ofsuch light. The second lenslet 444 receives at least a portion of thelight transmitted and/or shaped by the first lenslet and transmitsand/or shapes a portion of such light. The third lenslet 446 receives atleast a portion of the light transmitted and/or shaped by the secondlenslet and transmits and/or shapes a portion of such light to producethe image in the image area at the image plane 449.

FIGS. 15A-15F are schematic representations of some other types oflenses that may be employed. More particularly, FIGS. 15A-15E areschematic representations of other lenses 450-458 that include a stackhaving three lenslets 450A-450C, 452A-452C, 454A-454C, 456A-456C,458A-458C. FIG. 15F is a schematic representation of a lens 460 havingonly one lens element. It should be understood however, that an opticsportion may have any number of components and configuration.

FIGS. 16A-16C are representations of one embodiment of a sensor array,e.g., sensor array 310A, and circuits connected thereto, e.g., 470-476.The purpose of the sensor array, e.g., sensor array 310A, is to capturelight and convert it into one or more signals (e.g., electrical signals)indicative thereof, which are supplied to one or more of the circuitsconnected thereto, for example as described below. Referring to FIG.16A, the sensor array includes a plurality of sensor elements such asfor example, a plurality of identical photo detectors (sometimesreferred to as “picture elements” or “pixels”), e.g., pixels 480_(1,1)-480 _(n,m). The photo detectors, e.g., photo detectors 480_(1,1)-480 _(n,m), are arranged in an array, for example a matrix typearray. The number of pixels in the array may be, for example, in a rangefrom hundreds of thousands to millions. The pixels may be arranged forexample, in a 2 dimensional array configuration, for example, having aplurality of rows and a plurality of columns, e.g., 640×480, 1280×1024,etc. However, the pixels can be sized and dimensioned as desired, andcan be distributed in any desired pattern. Pixels that are distributedwithout any regular pattern could even be used. Referring to FIG. 16B, apixel, e.g., pixel pixels 480 _(1,1,)may be viewed as having dimensions,e.g., x and y dimensions, although it should be recognized that thephoton capturing portion of a pixel may or may not occupy the entirearea of the pixel and may or may not have a regular shape. In someembodiments, the sensor elements are disposed in a plane, referred toherein as a sensor plane. The sensor may have orthogonal sensorreference axes, including for example, an x axis, a y axis, and a zaxis, and may be configured so as to have the sensor plane parallel tothe xy plane XY and directed toward the optics portion of the camerachannel. Each camera channel has a field of view corresponding to anexpanse viewable by the sensor array. Each of the sensor elements maybe, for example, associated with a respective portion of the field ofview.

The sensor array may employ any type of technology, for example, but notlimited to MOS pixel technologies (meaning that one or more portions ofthe sensor are implemented in “Metal Oxide Semiconductor” technology),charge coupled device (CCD) pixel technologies or combination of both(hybrid) and may comprise any suitable material or materials, including,for example, silicon, germanium and/or combinations thereof. The sensorelements or pixels may be formed in any suitable manner.

In operation, the sensor array, e.g., sensor array 310A, is exposed tolight, for example, on a sequential line per line basis (similar toscanner) or globally (similar to conventional film camera exposure).After being exposed to light for certain period of time (exposure time),the pixels, e.g., pixels 480 _(1,1)-480 _(n,m), may be read out, e.g.,on a sequential line per line basis.

In some embodiments, circuitry sometimes referred to as column logic,e.g., column logic 470, is used to read the signals from the pixels,e.g., pixels 480 _(1,1)-480 _(n,m). Referring to FIG. 16C, a schematicrepresentation of a pixel circuit, in some of such embodiments, thesensor elements, e.g., pixel 480 _(1,1), may be accessed one row at atime by asserting one of the word lines, e.g., word line 482, which runhorizontally through the sensor array, e.g., sensor array 310A. Data maypassed into and/or out of the sensor elements, e.g., pixel 480 _(1,1),via bit lines which run vertically through the sensor array, e.g.,sensor array 310A.

It should be recognized that pixels are not limited to theconfigurations shown in FIGS. 16A-16C. As stated above, each of the oneor more sensor arrays may have any configuration (e.g., size, shape,pixel design).

The sensor arrays, e.g., sensor arrays 310A-310D, may or may not beidentical to one another. In some embodiments, for example, the sensorarrays are identical to one another. In some other embodiments, one ormore of the sensor arrays are different, in one or more respects, fromone or more of the other sensor arrays. For example, in someembodiments, one or more of the characteristics (for example, but notlimited to, its type of element(s), size (for example, surface area),and/or performance) of one or more of the sensor arrays is tailored tothe respective optics portion and/or to help achieve a desired result.For example, if a particular camera channel is dedicated to a particularcolor (or band of colors) or wavelength (or band of wavelengths), thesensor array for that camera channel may be adapted to have asensitivity that is higher to that particular color (or band of colors)or wavelength (or band of wavelengths) than other colors or wavelengthsand/or to sense only that particular color (or band of colors) orwavelength (or band of wavelengths). In some of such embodiments, thedesign, operation, array size (for example, surface area of the activeportion of the array), shape of the pixel of a sensor array (forexample, the shape of the active area (surface area of the pixel that issensitive to light) of the pixel) and/or pixel size of a sensor array(for example, the active area of the surface of the pixel) isdetermined, selected, tailored and/or optimized for the respectivewavelength or bands of wavelengths to which the camera channels arededicated. It should be understood, however, that any otherconfigurations may also be employed. Each of the one or more sensorarrays may have any configuration (e.g., size and shape).

As described herein, each sensor array may be, for example, dedicated toa specific band of light (visible and/or invisible), for example, onecolor or band of colors. If so, each sensor array may be tuned so as tobe more efficient in capturing and/or processing an image or images inits particular band of light.

In this embodiment, the well depth of the photo detectors across eachindividual array is the same, although in some other embodiments, thewell depth may vary. For example, the well depth of any given array canreadily be manufactured to be different from that of other arrays of thesensor subsystem. Selection of an appropriate well depth could depend onmany factors, including most likely the targeted band of visiblespectrum. Since each entire array is likely to be targeted at one bandof visible spectrum (e.g., red) the well depth can be designed tocapture that wavelength and ignore others (e.g., blue, green).

Doping of the semiconductor material in the color specific arrays mayenhance the selectivity of the photon absorption for color specificwavelengths.

In some embodiments, the pixels may be responsive to one particularcolor or band of colors (i.e., wavelength or band of wavelengths). Forexample, in some such embodiments, the optics portion may include lensesand/or filters that transmit only the particular color or band of colorsand/or attenuate wavelength or band of wavelengths associated with othercolors or band of colors. Is some others of such embodiments, a colorfilter and/or color filter array is disposed over and/or on one or moreportions of one or more sensor arrays. In some other embodiments, thereis no color filter or color filter array disposed on any of the sensorarrays. In some embodiments, the sensor array separates colors or bandsof colors. In some such embodiments, the sensor array may be providedwith pixels that have multiband sensing capability, e.g., two or threecolors. For example, each pixel may comprise two or three photodiodes,wherein a first photodiode is adapted to detect a first color or firstband of colors, a second photodiode is adapted to detect a second coloror band of colors and a third photodiode is adapted to detect a thirdcolor or band of colors. One way to accomplish this is to provide thephotodiodes with different structures/characteristics that make themselective, such that first photodiode has a higher sensitivity to thefirst color or first band of colors than to the second color or band ofcolors, and the second photodiode has a higher sensitivity to the secondcolor or second band of colors than to the first color or first band ofcolors. Another way is to dispose the photodiodes at different depths inthe pixel, which takes advantage of the different penetration andabsorption characteristics of the different colors or bands of colors.For example, blue and blue bands of colors penetrate less (and are thusabsorbed at a lesser depth) than green and green bands of colors, whichin turn penetrate less (and are thus absorbed at a lesser depth) thanred and red bands of colors. In some embodiments, such a sensor array isemployed even though the pixels may see only one particular color orband of colors, for example, to in order to adapt such sensor array tothe particular color or band of colors. Indeed, a layer of material thatattenuates certain wavelengths and passes other wavelengths may bedisposed on or integrated into the surface of the photodiode. In thisway, each pixel function as a plurality of photodiodes that is adaptedto sense multiple wavelengths.

FIG. 17A is a schematic plan view of a portion of a sensor array, e.g.,a portion of sensor array 310A, in accordance with one embodiment of thepresent invention. The portion of the array includes six unit cells,e.g., cells 490 _(i,j)-490 _(i+2,j+1). Each unit cell has a pixelregion, e.g., unit cell 490 _(i+2,j+1) has a pixel region 492_(i+2,j+1). The pixel region may be, for example, but is not limited to,a p implant region. The sensor elements, e.g., pixels 492 _(i,j)-492_(i+2,j+1), may be accessed one row at a time by asserting one of theword lines, e.g., word lines 494, which may run, for example,horizontally through the sensor array, e.g., sensor array 310A. Powermay be provided on power lines, e.g., power lines 496, which may forexample, run vertically through the sensor array. Data may passed intoand/or out of the sensor elements, e.g., pixels 492 _(i,j)-492_(i+2,j+1), via bit lines, e.g., bit lines 498, which may run, forexample, vertically through the sensor array, e.g., sensor array 310A.Reset may be initiated via reset lines, e.g., reset lines 500, which mayrun, for example, horizontally through the sensor array.

In some embodiments, each sensor array has 1.3 M pixels. In suchembodiments, three camera channels may provide an effective resolutionof about 4 M pixels. Four camera channels may provide an effectiveresolution of about 5.2 M pixels.

In some other embodiments, each sensor array has 2 M pixels. In suchembodiments, three camera channels may provide an effective resolutionof about 6 M pixels. Four camera channels may provide an effectiveresolution of about 8 M pixels.

It should be recognized that the sensor arrays are not limited to thedesign shown in FIG. 17A. As stated above, each of the one or moresensor arrays may have any configuration (e.g., size, shape, pixeldesign).

FIG. 17B is exemplary schematic cross section of the implant portion ofa pixel having a single well to capture all wavelengths.

For example, FIG. 17C is exemplary schematic cross section of an implantportion of a pixel having a well formed “deep” in the semiconductor (forexample, silicon) such that the depth of the implant is adapted orsuitable to improve capture or collect of light having wavelengths inthe range associated with the color red (among others). As such, theembodiment illustrated in FIG. 17C includes a deep implant formation ofthe junction to create a high efficiency red detector in which photonsare collected, detected or captured deep in the semiconductor. In thisembodiment, it may be advantageous to employ a color filter or opticalfiltration of the light prior to incidence on the pixel in order tosubstantially attenuate light having wavelengths associated with colorsother than red (photons having wavelengths in the range associated withred).

The well depth of the pixel or photo detector may be predetermined,selected and/or designed to tune the response to the photo detector. Inthis regard, with reference to FIG. 17D, a pixel “tuned” to capture,collect or respond to photons having wavelengths in the range associatedwith the color blue is illustrated. The exemplary schematic crosssection of an implant portion of a pixel includes a well formed “nearthe surface” in the semiconductor (for example, silicon) such that thedepth of the implant is adapted or suitable to improve capture orcollect of light having wavelengths in the range associated with thecolor blue. Accordingly, relative to FIG. 17C, a shallow junction isformed in the semiconductor which is optimized for collecting, detectingor capturing wavelengths in the range associated with the color bluenear the surface of the detector (relative to FIG. 17C). As such, inthis embodiment, a filter may be omitted due to selectively implantingthe region at a particular depth. That is, filter material may beunnecessary as both green and red photons pass through the collectionregion collecting, detecting or capturing mainly the blue signal(photons having wavelengths in the range associated with the colorblue).

With reference to FIG. 17E, the pixel or photo detector may be “tuned”to capture, collect or respond to photons having wavelengths primarilyin the range associated with the color red. Here, the well region isformed and/or confined at a depth that is associated primarily withwavelengths of the color red.

With reference to FIG. 17F, the pixel or photo detector may be “tuned”to capture, collect or respond to photons having wavelengths primarilyin the range associated with the color green. Here, the well region isformed and/or confined at a depth that is associated primarily withwavelengths of the color green.

Notably, the pixel or photo detector may be “tuned” to capture, collector respond to photons having wavelengths primarily in the rangeassociated with any color. In this regard, the well region of the pixelor photo detector is formed and/or confined at a depth that isassociated primarily with wavelengths of the color to be captured orcollected. In these embodiments, the specific regions for collection canbe formed by buried junctions within the semiconductor base material. Inthis case by varying the buried junction depth and shape, wavelengthselectivity can be achieved. Together with the optical path furtherselectivity and wavelength responsivity can allow for single or multipleband pass detectors.

The pixel or photo detector may be “tuned” to capture, collect orrespond to photons having wavelengths primarily in the range associatedwith more than one color. For example, with reference to FIG. 17G, afirst pixel (located on the left) includes well regions formed and/orconfined at a depth that are associated primarily with wavelengths ofthe colors red (deep) and blue (more shallow). As such, this pixel orphoto detector is “tuned” to capture or collect incident photons havingwavelengths primarily in the range associated with two colors. The pixelon the right includes a well region formed and/or confined at a depththat is associated primarily with wavelengths of one color, here green.The sensor array may include one, some or all of the pixels (located onthe left or the right). Moreover, the sensor array may include a patternof both types of pixels.

Notably, the pixel or photo detector may be “tuned” to capture, collector respond to photons having wavelengths primarily in the rangeassociated with any two or more colors (provided that such colors aresufficiently spaced to permit appropriate sensing). (See for example,FIG. 17H—blue and green sensed via the pixel located on the left andgreen and red sensed via the pixel located on the right).

There are many embodiments related to tuning the depth of the welland/or region of the pixel or photo detector, for example,

λ3/λ2/λ1 (e.g. R/G/B) color filter array on individual pixels

λ3/λ2/λ1 (e.g. R/G/B) photodiodes in individual pixels

λ3/λ1 (e.g. R/B) photodiodes in one pixel, λ2 (e.g. G) in one pixel

λ3/λ2/λ1 (e.g. R/G/B) photodiodes in one pixel

λ4/λ2 (e.g. R/G1) photodiodes in one pixel, λ3/λ1 (e.g. G2/B) in onepixel

λ4/λ3/λ2/λ1 (e.g. R/G2/G1/B)color filter arrays on individual pixels

λ4/λ3/λ2/λ1 (e.g. R/G2/G1/B) photodiodes in one pixel

λ4/λ3/λ2/λ1 (e.g. R/G2/G1/B) photodiodes in individual pixels

Note: wavelength bands from λ1 to λ4 represent increasing wavelengthsand can range from the UV to IR (e.g. 200-1100 nm for siliconphotodiodes)

All embodiments for related to tuning the depth of the well and/orregion of the pixel or photo detector, are intended to fall within thescope of the present invention and, as such, may be implemented in anyof the embodiments described and illustrated herein.

In sum, since each array of photo detectors is separate from the other,and unlike conventional arrays which can only be processed in a likemanner due to the proximity of adjacent photo detectors, various implantand junction configurations may be achieved by this invention. Using oneor more of the techniques and/or embodiments described above or acombination of filters and wavelength specific detectors, various photodetector topologies can be achieved.

The configuration of a sensor array (e.g., number, shape, size type andarrangement of sensor elements) may impact the characteristics of thesensed images. For example, FIGS. 18A-18B are explanatoryrepresentations depicting an image being captured by a portion of asensor array, e.g., 310A. More particularly, FIG. 18A is a explanatoryview of an image of an object (a lightning bolt) striking a portion ofthe sensor array. In this example, the photon capturing portions (oractive area), e.g., photon capturing portion 502, of the sensor elementsare represented generally represented by circles although in practice, apixel can have any shape including for example, an irregular shape. Forpurposes of this example, photons that strike the photon capturingportion or active area of the pixel or photo detector (e.g., photonsthat strike within the circles XX) are sensed and/or captured thereby.FIG. 18B shows the portion of the photons, e.g., portion 504, that arecaptured by the sensor in this example. Photons that do not strike thesensor elements (e.g., photons that striking outside circles XX) are notsensed/captured.

FIGS. 19A-19B are explanatory representations depicting an image beingcaptured by a portion of a sensor, e.g., sensor array 310A, that hasmore sensor elements and closer spacing of such elements than isprovided in the sensor of FIG. 18A. More particular, FIG. 19A shows animage of an object (a lightning bolt) striking the sensor. For purposesof this example, photons that strike the photon capturing portion, e.g.,photon capturing portion 506, are sensed and/or captured thereby. FIG.19B shows the portions of the photons, e.g., portion 508, that arecaptured by the sensor in this example. Notably, the sensor of FIG. 19Acaptures more photons than the sensor of FIG. 18A.

FIGS. 20A-20B are schematic representations of a relative positioningprovided for an optics portion, e.g., optics portion 330A, and arespective sensor array, e.g., sensor array 310A, in some embodiments.In that regard, it should be understood that, although, FIGS. 20A-20Bshows the optics portion having an axis, e.g., axis 510A, aligned withan axis, e.g., axis 512A, of the sensor array, some embodiments may notemploy such an alignment. In addition, in some embodiments, the opticsportion and/or the sensor array may not have an axis.

FIG. 21 is a schematic representation of a relative positioning providedfor four optics portions, e.g., optics portions 330A-330D, and foursensor arrays, e.g., sensor arrays 310A-310D, in some embodiments.Although FIG. 21 shows each of the optics portions, e.g., optics portion330B, having an axis, e.g., axis 510B, aligned with an axis, e.g., axis512B, of the respective sensor array, e.g., sensor array 310B, it shouldbe understood that some embodiments may not employ such an alignment. Inaddition, in some embodiments, the one or more of the optics portionsand/or one or more of the sensor arrays may not have an axis.

In some embodiments, the optics portion is generally about the same sizeas the respective sensor array, and may therefore differ from oneanother in size and shape depending upon the dimensions of theunderlying array. There is, however, no requirement that a given opticsportion cover all, or only, the underlying array. In some alternativeembodiments an optics portion could cover only a portion of an arrayand/or could extend beyond the array.

FIGS. 22A-22B are a schematic plan view and a schematic cross sectionalview, respectively, of one embodiment of an image device 520 in or onwhich one or more sensor arrays, e.g., sensor arrays 310A-310D, may bedisposed and/or integrated, and the image areas of the respective opticsportions, e.g., optics portions 330A-330D, in accordance with oneembodiment of the present invention. In this embodiment, the imagedevice 520 has first and second major surfaces 522, 524 and an outerperimeter defined by edges 526, 528, 530 and 532. The image device 520defines the one or more regions, e.g., regions 534A-534D, for the activeareas of the one or more sensor arrays, e.g., sensor arrays 310A-310D,respectively. The image device further defines one or more regions,e.g., regions 536A-536D, respectively, and 538A-538D, respectively, forthe buffer and/or logic associated with the one or more sensor arrays,e.g., sensor arrays 310A-310D, respectively.

The image device may further define one or more additional regions, forexample, regions 540, 542, 544, 546 disposed in the vicinity of theperimeter of the image device (e.g., extending along and adjacent toone, two, three or four of the edges of the image device) and/or betweenthe regions for the sensor arrays. In some embodiments, one or moreelectrically conductive pads, e.g., pads 550, 552, 554, 556, one or moreportions of the processor, one or more portions of additional memory,and/or any other types of circuitry or features may be disposed in oneor more of these regions, or portions thereof. One or more of such padsmay be used in supplying one or more electrical signals and/or from oneor more circuits on the image device to one or more other circuitslocated on or off of the image device.

In some embodiments, the major outer surface defines one or more supportsurfaces to support one or more portions of a support, e.g., support320. Such support surfaces may be disposed in any region, or portionthereof, e.g., regions 540-546, however in some embodiments, it isadvantageous to position the support surfaces outside the active areasof the sensor array so as not to interfere with the capture of photonsby pixels in such areas.

The one or more optics portions, e.g., optics portions 330A-330D,produce image areas, e.g., image areas 560A-560D, respectively, at animage plane.

The image device, sensor arrays and image areas may each have anysize(s) and shape(s). In some embodiments, the image areas are generallyabout the same size as the respective sensor arrays, and therefore, theimage areas may differ from one another in size and shape depending uponthe dimensions of the underlying sensor arrays. Of course, there is norequirement that an image area cover all, or only, the underlying array.In alternative embodiments an image area could cover only a portion ofan array, and could extend beyond the array.

In this embodiment, the image areas, e.g., image areas 560A-560D, extendbeyond the outer perimeter of the sensor arrays, e.g., sensor arrays310A-310D, respectively. The image device has a generally square shapehaving a first dimension 562 equal to about 10 mm and a second dimension564 equal to about 10 mm, with each quadrant having a first dimension566 equal to 5 mm and a second dimension 568 equal to 5 mm. Each of theimage areas has a generally circular shape and a width or diameter 570equal to about 5 millimeters (mm). Each of the active areas has agenerally rectangular shape having a first dimension 572 equal to about4mm and a second dimension 574 equal to about 3mm. The active area, maydefine for example, a matrix of 1200×900 pixels (i.e., 1200 columns, 900rows).

FIGS. 23A-23B are a schematic plan view and a schematic cross sectionalview, respectively, of the image device and image areas in accordancewith another embodiment. In this embodiment, the image device 520 hasone or more pads, e.g., 550-556, disposed in a configuration that isdifferent than the configuration of the one or more pads in theembodiments shown above. The image device 520, sensor arrays, and imageareas 560A-560D may have, for example, the same shape and dimensions asset forth above with respect to the embodiment of the image device shownin FIGS. 22A-22B.

FIGS. 24A-24B are a schematic plan view and a schematic cross sectionalview, respectively, of the image device 520 and image areas inaccordance with another embodiment. In this embodiment, the image device520 has a vertically extending region, disposed between the sensorarrays, that is narrower than a vertically extending region, disposedbetween the sensor arrays, in the embodiment of the image device shownin FIGS. 22A-22B. Horizontally extending regions 542, 546, disposedalong the perimeter are wider than horizontally extending regions 542,546, disposed along the perimeter of the image device 520 shown in FIGS.22A-22B. The image device 520 may have, for example, the same shape anddimensions as set forth above with respect to the embodiment of theimage device shown in FIGS. 22A-22B.

FIGS. 25A-25B are a schematic plan view and a schematic cross sectionalview, respectively, of the image device 520 and image areas, e.g., imageareas 560A-560D, in accordance with another embodiment. In thisembodiment, the image areas, e.g., image areas 560A-560D, do not extendbeyond the outer perimeter of the sensor arrays, e.g., sensor arrays310A-310D, respectively. The image device 520 and the sensor arrays mayhave, for example, the same shape and dimensions as set forth above withrespect to the embodiment of the image device 520 shown in FIGS.22A-22B.

FIGS. 26A-26B are a schematic plan view and a schematic cross sectionalview, respectively, of the image device and image areas in accordancewith another embodiment. In this embodiment, regions 540-546 disposedbetween the sensor arrays and the edges of the image device are widerthan the regions 540-546 disposed between the sensor arrays and the edgeof the image device in the embodiments of FIGS. 22A-22B. Such regionsmay be used, for example, for one or more pads, one or more portions ofthe processor, as a seat and/or mounting region for a support and/or anycombination thereof.

In addition, in this embodiment, a horizontally extending region 564disposed between the sensor arrays is wider than the horizontallyextending region 546 between the sensor arrays in the embodiment ofFIGS. 22A-22B. Such region 546 may be used, for example, for one or morepads, one or more portions of the processor, as a seat and/or mountingregion for a support and/or any combination thereof. The image deviceand the sensor arrays may have, for example, the same shape anddimensions as set forth above.

As with each of the embodiments disclosed herein, this embodiment may beemployed alone or in combination with one or more of the otherembodiments disclosed herein, or portions thereof.

To that effect, for example, FIGS. 27A-27B are a schematic plan view anda schematic cross sectional view, respectively, of the image device 540and image areas 560A-560D in accordance with another embodiment. Thisembodiment of the image device 520 and image areas 560A-560D is similarto the embodiment of the image device and image areas shown in FIGS.26A-26B, except that the image areas, e.g., image areas 560A-560D, donot extend beyond the outer perimeter of the sensor arrays, e.g., sensorarrays 310A-310D, respectively.

FIG. 28A is a schematic perspective view of a support 320 in accordancewith another embodiment of the present invention. The support 320 mayhave any configuration and may comprise, for example, but is not limitedto, a frame. FIGS. 28B-28D are enlarged cross sectional views of thesupport 320. Referring to FIGS. 28A-28D, the optics portions of the oneor more camera channels, e.g., optics portions 330A-330D, are supportedby one or more supports, e.g., the support 320, which position(s) eachof the optics portions in registration with a respective sensor array,at least in part. In this embodiment, for example, optics portion 330Ais positioned in registration with sensor array 310A. Optics portion330B is positioned in registration with sensor array 310B. Opticsportion 330C is positioned in registration with sensor array 310C.Optics portion 330B is positioned in registration with sensor array310B. Optics portion 330D is positioned in registration with sensorarray 310D.

In some embodiments, the support 320 may also help to limit, minimizeand/or eliminate light “cross talk” between the camera channels and/orhelp to limit, minimize and/or eliminate “entry” of light from outsidethe digital camera apparatus.

In some embodiments, the support 320 defines one or more supportportions, e.g., four support portions 600A-600D, each of which supportsand/or helps position a respective one of the one or more opticsportions. In this embodiment, for example, support portion 600A supportsand positions optics portion 330A in registration with sensor array310A. Support portion 600B supports and positions optics portion 330B inregistration with sensor array 310B. Support portion 600C supports andpositions optics portion 330C in registration with sensor array 310C.Support portion 600D supports and positions optics portion 330D inregistration with sensor array 310D.

In this embodiment, each of the support portions, e.g., support portions600A-600D, defines an aperture 616 and a seat 618. The aperture 616defines a passage for the transmission of light for the respectivecamera channel. The seat 618 is adapted to receive a respective one ofthe optics portions (or portion thereof) and to support and/or positionthe respective optics portion, at least in part. In this regard, theseat 618 may include one or more surfaces (e.g., surfaces 620, 622)adapted to abut one or more surfaces of the optics portion to supportand/or position the optics portion, at least in part, relative to thesupport portion and/or one or more of the sensor arrays 310A-310D. Inthis embodiment, surface 620 is disposed about the perimeter of theoptics portion to support and help position the optics portion in the xdirection and the y direction). Surface 622 (sometimes referred toherein as “stop” surface) positions or helps position the optics portionin the z direction.

The position and/or orientation of the stop surface 622 may be adaptedto position the optics portion at a specific distance (or range ofdistance) and/or orientation with respect to the respective sensorarray. In this regard, the seat 618 controls the depth at which the lensis positioned (e.g., seated) within the support 320. The depth may bedifferent for each lens and is based, at least in part, on the focallength of the lens. For example, if a camera channel is dedicated to aspecific color (or band of colors), the lens or lenses for that camerachannel may have a focal length specifically adapted to the color (orband of colors) to which the camera channel is dedicated. If each camerachannels is dedicated to a different color (or band of colors) than theother camera channels, then each of the lenses may have a differentfocal length, for example, to tailor the lens to the respective sensorarray, and each of the seats have a different depth.

Each optics portion may be secured in respective seat 618 in anysuitable manner, for example, but not limited to, mechanically (e.g.,press fit, physical stops), chemically (e.g., adhesive), electronically(e.g., electronic bonding) and/or combination thereof. The seat 618 mayinclude dimensions adapted to provide a press fit for the respectiveoptics portion.

The aperture (or portions thereof) may have any configuration (e.g.,shape and/or size) including for example, cylindrical, conical,rectangular, irregular and/or any combination thereof. The configurationmay be based, for example, on the desired configuration of the opticalpath, the configuration of the respective optical portion, theconfiguration of the respective sensor array and/or any combinationthereof.

It should be understood that the support 320 may or may not have exactlyfour support portions, e.g., support portions 600A-600D. In someembodiments, for example, the support includes fewer than four supportportions (e.g., one, two or three support portions) are used. In someother embodiments, the support includes more than four support portions.Although the support portions, 630A-630D are shown as being identical toone another, this is not required. Moreover, in some embodiments, one ormore of the support portions may be isolated at least in part from oneor more of the other support portions. For example, the support 320 mayfurther define clearances or spaces that isolate the one or more innersupport portions, in part, from one or more of the other supportportions.

The support 320 may comprise any type of material(s) and may have anyconfiguration and/or construction. In some embodiments, for example, thesupport 320 comprises silicon, semiconductor, glass, ceramic, plastic,or metallic materials and/or a combination thereof. If the support 320has more than one portion, such portions may be fabricated separate fromone another, integral with one another and/or any combination thereof.If the support defines more than one support portion, each of suchsupport portions, e.g., support portions 600A-600D, may be coupled toone, some or all of the other support portions, as shown, or completelyisolated from the other support portions. The support may be a soliddevice that may offer a wide range of options for manufacturing andmaterial, however other forms of devices may also be employed. In someembodiments, for example, the support 320 comprises a plate (e.g., athin plate) that defines the one or more support portions, with theapertures and seats being formed by machining (e.g., boring) or anyother suitable manner. In some other embodiments, the support 320 isfabricated as a casting with the apertures defined therein (e.g., usinga mold with projections that define the apertures and seats of the oneor more support portions).

In some embodiments, the lens and support are manufactured as a singlemolded component. In some embodiments the lens may be manufactured withtabs that may be used to form the support.

In some embodiments, the support 320 is coupled and/or affixed directlyor indirectly, to the image device. For example, the support 320 may bedirectly coupled and affixed to the image device (e.g., using adhesive)or indirectly coupled and/or or affixed to the image device via anintermediate support member (not shown).

The x and y dimensions of the support 320 may be, for example,approximately the same (in one or more dimensions) as the image device,approximately the same (in one or more dimensions) as the arrangement ofthe optics portions 330A-330D and/or approximately the same (in one ormore dimensions) as the arrangement of the sensor arrays 310A-310D. Oneadvantage of such dimensioning is that it helps keep the x and ydimensions of the digital camera apparatus as small as possible.

In some embodiments, it may be advantageous to provide the seat 618 witha height A that is the same as the height of a portion of the opticsthat will abut the stop surface 620. It may be advantageous for the stopsurface 622 to be disposed at a height B (e.g., the distance between thestop surface 622 and the base of the support portion) that is at leastas high as needed to allow the seat 618 to provide a firm stop for anoptics portion (e.g., the lens) to be seated thereon. The width ordiameter C of the portion of the aperture 616 disposed above the heightof the stop surface 622 may be based, for example, on the width ordiameter of the optics portion (e.g., the lens) to be seated therein andthe method used to affix and/or retain that optics portion in the seat618. The width of the stop surface 622 is preferably large enough tohelp provide a firm stop for the optics portion (e.g., the lens) yetsmall enough to minimize unnecessary blockage of the light transmittedby the optics portion. It may be desirable to make the width or diameterD of the portion of the aperture 616 disposed below the height of thestop surface 622 large enough to help minimize unnecessary blockage ofthe light transmitted by the optics portion. It may be desirable toprovide the support with a height E equal to the minimum dimensionneeded to result in a support sturdy enough to support the one or moreoptics portions to be seated therein, in view of the considerationsabove, and may be advantageous to space the one or more apertures616A-616D of the one or more support portions 600A-600D by a distance Fthat is as small as possible yet large enough that the support will besturdy enough to support the optics portions to be seated therein. Thesupport may have a length J and a width K.

In some embodiments, it is desirable to provide the seat 618 with aheight A equal to 2.2 mm, to provide the stop surface 622 at a height Bin the range of from 0.25 mm to 3 mm, to make the width or diameter C ofthe portion of the aperture above the height B of the stop surface 622equal to approximately 3 mm, to make the width or diameter D of thelower portion of the aperture approximately 2.8 mm, to provide thesupport portion with a height E in the range from 2.45 mm to 5.2 mm andto space the apertures apart by a distance F of at least 1 mm. In someof such embodiments, it may be desirable to provide the support with alength J equal to 10 mm and a width K equal to 10 mm. In some otherembodiments, it may be desirable to provide the support with a length Jequal to 10 mm and a width K equal to 8.85 mm.

In some embodiments, one or more of the optics portions, e.g., opticsportion 330A, comprises a cylindrical type of lens, e.g., a NT45-090lens manufactured by Edmunds Optics, although this is not required. Suchlens has a cylindrical portion with a diameter G up to 3 millimeters(mm) and a height H of 2.19 mm. In such embodiments, it may be desirableto employ a support having the dimensions and ranges set forth in theparagraph above.

In some embodiments, the support has a length J equal to 10 mm and awidth K equal to 10 mm. In some other embodiments, it may be desirableto provide the support with a length J equal to 10 mm and a width Kequal to 8.85 mm.

FIG. 29A is a schematic cross sectional view of a support 320 and opticsportions, e.g., 330A-330D, seated therein in accordance with anotherembodiment. In this embodiment, the optics portions have an orientationthat is inverted compared to the orientation of the optics portions inthe embodiment of FIGS. 7A-7C.

FIG. 29B is a schematic cross sectional view of a support and opticsportions, e.g., 330A-330D, seated therein in accordance with anotherembodiment. In this embodiment, each of the optics portions includes asingle lens element having a shank portion 702A-702D, respectively. Thesupport 320 has an orientation that is inverted compared to theorientation of the support in the embodiment of FIGS. 6A-6C, such thatthe optics portions are seated on stop surfaces 622A-622D, respectively,that face in a direction away from the sensor arrays (not shown).

It should be understood that the features of the various embodimentsdescribed herein may be used alone and/or in any combination thereof.

FIGS. 30A-30D show a support 320 having four support portions 600A-600Deach defining an aperture, e.g., aperture 616A-616D, for a respectiveoptics portion, wherein the seat, e.g., seat 618A, defined by one ormore of the support portions, e.g., support portion 600A, is disposed ata depth 710A that is different than the depths, e.g., depth 710C, of theseat, e.g., seat 618C, of one or more other support portions, forexample, to adapt the one or more support portions to the focal lengthof the respective optics portions. As stated above, the position and/ororientation of the stop surface 622 may be adapted to position theoptics portion at a specific distance (or range of distance) and/ororientation with respect to the respective sensor array. In this regard,the seat 618 controls the depth at which the lens is positioned (e.g.,seated) within the support 320. In some embodiments, one of the opticsportions is adapted for blue light or a band of blue light and anotherone of the optics portions is adapted for red light or a band of redlight, however other configurations may also be employed.

FIGS. 31A-31D show a support 320 having four support portions 600A-600Deach defining an aperture 616A-616D and a seat 618A-618D, respectively,for a respective optics portion, wherein the aperture, e.g., aperture616A, of one or more of the support portions, e.g., support portion600A, has a diameter 714A that is less than the diameter 714C of theaperture 616 of one or more of the other support portions, e.g., supportportion 600C.

As with each of the embodiments disclosed herein, this embodiment may beemployed alone or in combination with one or more of the otherembodiments disclosed herein, or portions thereof. In that regard, insome embodiments, the seat defined by one or more of the supportportions is at a depth that is different than the depths of the seats ofthe other support portions so as to adapt such one or more supportportions to the focal length of the respective optics portions, as inthe embodiment of the support shown in FIGS. 30A-30D.

In some embodiments, one of the optics portions is adapted for bluelight or a band of blue light and another one of the optics portions isadapted for red light or a band of red light, however otherconfigurations may also be employed.

FIG. 32 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith one embodiment of the present invention. In this embodiment, theone or more optics portions, e.g., optics portions 330A-330D are seatedin and/or affixed to the support 320. The support 320 is disposedsuperjacent a first bond layer 722, which is disposed superjacent animage device, e.g., image device 520, in or on which the one or moresensor portions, e.g., sensor portions 310A-310D, are disposed and/orintegrated. The image device 520 is disposed superjacent a second bondlayer 724 which is disposed superjacent the printed circuit board 110.

The printed circuit board includes a major outer surface 730 thatdefines a mounting region on which the image device is mounted. Themajor outer surface 730 may further define and one or more additionalmounting regions (not shown) on which one or more additional devicesused in the digital camera may be mounted. One or more pads 732 areprovided on the major outer surface 730 of the printed circuit board toconnect to one or more of the devices mounted thereon.

The image device 520 includes the one or more sensor arrays, e.g.,sensor arrays 310A-310D, and one or more electrically conductive layers.In some embodiments, the image device further includes one, some or allportions of the processor for the digital camera apparatus. The imagedevice 520 further includes a major outer surface 740 that defines amounting region on which the support 320 is mounted.

The one or more electrically conductive layers may be patterned todefine one or more pads 742 and one or more traces (not shown) thatconnect the one or more pads to one or more of the one or more sensorarrays. The pads 742 are disposed, for example, in the vicinity of theperimeter of the image device 520, for example, along one, two, three orfour sides of the image device. The one or more conductive layers maycomprise, for example, copper, copper foil, and/or any other suitablyconductive material(s).

A plurality of electrical conductors 750 may connect one or more of thepads 742 on the image device 520 to one or more of the pads 732 on thecircuit board 720. The conductors 750 may be used, for example, toconnect one or more circuits on the image device to one or more circuitson the printed circuit board.

The first and second bond layers 722, 724 may comprise any suitablematerial(s), for example, but not limited to adhesive, and may compriseany suitable configuration. The first and second bond layers 722, 724may comprise the same material(s) although this is not required. As usedherein, a bond layer may be continuous or discontinuous. For example, aconductive layer may be an etched printed circuit layer. Moreover, abond layer may or may not be planar or even substantially planar. Forexample, a conformal bond layer on a non-planar surface will benon-planar.

A plurality of optics portions, e.g., optics portions 330A-330D areseated in and/or affixed to the support.

In some embodiments, the digital camera apparatus 300 has dimensions ofabout 2.5 mm×6 mm×6 mm. For example, the thickness may be equal to about2.5 mm, the length may be equal to about 6 mm and the width may be equalto about 6 mm. In some of such embodiments, the digital camera apparatushas one or more sensor arrays having a total of 1.3 M pixels, althoughother configurations may be employed (e.g., different thickness, width,length and number of pixels).

In some embodiments, one or more of the circuits on the image device 520may communicate with one or more devices through one or more wirelesscommunication links. In some such embodiments, the image device 520 maydefine one or more circuits for use in such wireless communication linkand/or one or more mounting regions for one or more discrete devicesemployed in such wireless communication link(s).

The digital camera apparatus 300 may be assembled and mounted in anymanner. FIGS. 33A-33F shows one embodiment for assembling and mountingthe digital camera apparatus. Referring to FIG. 33A, initially, theimage device 520 is provided. Referring to FIG. 33B, a first bond layer722 is provided on one or more regions of one or more surfaces of theimage device 520. Such regions define one or more mounting regions forthe support. Referring to FIG. 33C, the support 520 is thereafterpositioned on the bond layer 722. In some embodiments, force may beapplied to help drive any trapped air out from between the image deviceand support. In some embodiments, heat and/or force may be applied toprovide conditions to activate and/or cure the bond layer to form a bondbetween the image device 520 and the support 320. Referring to FIG. 33D,one or more optics portions, e.g., optics portions 330A-330D maythereafter be seated in and/or affixed to the support 320. Referring toFIG. 33E, a bond layer 724 is provided on one or more regions of one ormore surfaces of the printed circuit board 720. Such regions define oneor more mounting regions for the digital camera apparatus 300. Referringto FIG. 33F, the digital camera apparatus 300 is thereafter positionedon the bond layer 724. One or more electrical conductors 750 may beinstalled to connect one or more of pads 742 on the image device to oneor more pads on circuit board 732.

In some embodiments, the electrical interconnect between componentlayers may be formed by lithography and metallization, bump bonding orother methods. Organic or inorganic bonding methods can be used to jointhe component layers. The layered assembly process may start with a“host” wafer with electronics used for the entire camera and/or eachcamera channel. Then another wafer or individual chips are aligned andbonded to the host wafer. The transferred wafers or chips can have bumpsto make electrical interconnect or connects can be made after bondingand thinning. The support substrate from the second wafer or individualchips is removed, leaving only a few microns material thickness attachedto the host wafer containing the transferred electronics. Electricalinterconnects are then made (if needed) between the host and the bondedwafer or die using standard integrated circuit processes. The processcan be repeated multiple times.

FIGS. 33G-33K are schematic views of a digital camera apparatus,mechanical mountings and electrical connections employed in accordancewith further embodiments of the present invention. More particularly,FIG. 33G is a schematic perspective view of the digital camera apparatus300. FIG. 33H is a schematic elevational view of the digital camera 300mounted to a major lower surface of a printed circuit board 720. One ormore electrical conductors 750 are used to connect pads 732 on theprinted circuit 720 to pads on the major outer surface of the imagedevice 520.

FIG. 33H is a schematic elevational view of the digital camera 300mounted to a major lower surface of a printed circuit board 720. Thesupport 320 is disposed in a through hole defined by the printed circuitboard. One or more electrical conductors 750 connect pads 732 on theprinted circuit 720 to pads on the major outer lower of the image device520.

FIG. 33I is a schematic elevational view of the digital camera 300mounted to a major lower surface of a printed circuit board 720. Thesupport 320 is disposed in a through hole defined by the printed circuitboard. A bump bond 752 connects one or more of the pads 742 on thesurface 740 of the image device 520 to pads 732 on the major lowersurface of the printed circuit board 720.

FIG. 33J is a schematic elevational view of the digital camera 300mounted to a major upper surface of a printed circuit board 720. One ormore electrical conductors 750 connect pads 732 on the printed circuit720 to pads 742 on the major outer surface 740 of image device 520.

FIG. 33I is a schematic elevational view of the digital camera 300mounted to a major lower surface of a printed circuit board 720. Thesupport 320 is disposed in a through hole defined by the printed circuitboard. A bump bond 752 connects one or more pads on a major lowersurface of the image device 520 to pads on the major upper surface ofthe printed circuit board 720.

In some embodiments, the manufacture of the optical stacks, and imagesensors are done on a single wafer, fabricated on separate wafers(perhaps up to two wafers: one for the IC, and one for optics) andbonded together at the wafer level. It is also possible to use pick andplace methods and apparatus to attach the optical assemblies to thewafer IC, or the image sensor die and other assemblies can be assembledindividually.

In embodiments that employ MEMS, manufacture of the optical stacks, MEMsand image sensors may be done on a single wafer, fabricated on separatewafers (perhaps up to three wafers: one for the IC, one for MEMs and onefor optics) and bonded together at the wafer level. It is also possibleto use pick and place methods and apparatus to attach the opticalassemblies and MEMs to the wafer IC, or the image sensor die and otherassemblies (MEMs and optical stack) can be assembled individually.

FIG. 34 is a schematic cross section view of a support that may beemployed to support one or more lenses having three lens elements, e.g.,lenses 410, 430 (FIGS. 11A-11B, 13A-13B), and to position such lenses inregistration with a respective sensor array, at least in part, inaccordance with another embodiment of the present invention. In thisembodiment, the support 320 defines one or more support portions, e.g.,four support portions 600A-600D, each of which supports and/or helpsposition a respective one of the one or more optics portions.

In some embodiments, the support may also help to limit, minimize and/oreliminate light “cross talk” between the camera channels and/or may alsohelp to limit, minimize and/or eliminate “entry” of light from outsidethe digital camera apparatus.

Each of the support portions 600A-600D defines an aperture 616 and aplurality of seats 618-1 to 618-3. More particularly, support portion600A defines an aperture 616A and seats 618-1A to 618-3C. Supportportion 600B defines an aperture 616B and seats 618-1B to 618-3B.Support portion 600C defines an aperture 616C and seats 618-1C to618-3C. Support portion 600D defines an aperture 616D and seats 618-1Dto 618-3D. Referring for example, to support portion 600A, the aperture616A defines a passage for the transmission of light for the respectivecamera channel. Each of the plurality of seats 618-1A to 618-3A isadapted to receive a respective one of the lenslets of the respectiveoptics portion (or portion thereof) and to support and/or position therespective lenslet, at least in part. In this regard, each of the seats618-1A to 618-3A may include one or more surfaces (e.g., surfaces 620-1Ato 620-3A, respectively, and surfaces 622-1A to 622-3A, respectively)adapted to abut one or more surfaces of the respective lenslet tosupport and/or position the lenslet, at least in part, relative to thesupport portion and/or one or more of the sensor arrays 310A-310D. Inthis embodiment, each of the surfaces 620-1A to 620-3A is disposed aboutthe perimeter of the respective lenslet to support and help positionsuch lenslet in the x direction and the y direction). Each of thesurfaces 622-1A to 622-3A (sometimes referred to herein as “stop”surface) positions or helps position the respective lenslet in the zdirection.

The positions and/or orientations of the stop surfaces 622-1A to 622-3Amay be adapted to position the respective lenslet at a specific distance(or range of distance) and/or orientation with respect to the respectivesensor array. In this regard, the seats 618-1A to 618-3A control thedepth at which each of the lenslets is positioned (e.g., seated) withinthe support. The depth may be different for each lenslet and is based,at least in part, on the focal length of the lens. For example, if acamera channel is dedicated to a specific color (or band of colors), thelens or lenses for that camera channel may have a focal lengthspecifically adapted to the color (or band of colors) to which thecamera channel is dedicated. If each camera channels is dedicated to adifferent color (or band of colors) than the other camera channels, theneach of the lenses may have a different focal length, for example, totailor the lens to the respective sensor array, and each of the seatshave a different depth.

In this embodiment, each of the support portions includes an elongatedportion adapted to help position the respective optics portions at adesired distance from the respective sensor arrays. In this embodiment,the elongated portions extend in an axial direction and define walls760, which in turn define the lower portions of apertures, respectively,which help limit, minimize and/or eliminate light “cross talk” betweenthe camera channels and help limit, minimize and/or eliminate “entry” oflight from outside the digital camera apparatus.

In some embodiments, the a spacer is provided, separately fabricatedfrom the support portions and adapted to be disposed between the supportportions and the one or more sensor arrays to help position the one ormore optics portions at a desired distance from the one or more sensorarrays. In some of such embodiments, the spacer and support collectivelydefine one or more passages for transmission of light, help to limit,minimize and/or eliminate light “cross talk” between the camera channelsand/or help to limit, minimize and/or eliminate “entry” of light fromoutside the digital camera apparatus.

The support 320 may comprise any type of material(s) and may have anyconfiguration and/or construction. In some embodiments, for example, thesupport 320 comprises silicon, semiconductor, glass, ceramic, plastic,or metallic materials and/or a combination thereof. If the support 320has more than one portion, such portions may be fabricated separate fromone another, integral with one another and/or any combination thereof.If the support defines more than one support portion, each of suchsupport portions, e.g., support portions 600A-600D, may be coupled toone, some or all of the other support portions, as shown, or completelyisolated from the other support portions.

The support 320 may be a solid device that may offer a wide range ofoptions for manufacturing and material, however other forms of devicesmay also be employed. In some embodiments, for example, the support 320comprises a plate (e.g., a thin plate) that defines the support and oneor more support portions, with the apertures and seats being formed bymachining (e.g., boring) or any other suitable manner. In some otherembodiments, the support 320 is fabricated as a casting with theapertures defined therein (e.g., using a mold with projections thatdefine the apertures and seats of the one or more support portions).

Each optics portion, e.g., optics portions 330A-330D, may be secured inthe respective seats in any suitable manner, for example, but notlimited to, mechanically (e.g., press fit, physical stops), chemically(e.g., adhesive), electronically (e.g., electronic bonding) and/or anycombination thereof. In some embodiments, each of the seats 618-1A to618C-3A has dimensions adapted to provide a press fit for the respectivelenslets.

Notably, the lenslets of the optics portions may be assembled into thesupport in any suitable manner.

FIGS. 35A-35C show one embodiment for assembling the lenslets of theoptics portions in the support. Referring to FIG. 35A, in thisembodiment, the support 320 is turned upside down and the bottomlenslets 410C, 430C of each lens 410, 430, respectively, is insertedinto the bottom of a respective aperture, seated in a respective seat618-3 and affixed thereto, if desired. Referring to FIG. 35B, thesupport 320 is thereafter turned right side up and the middle lenslet410B, 430B, of each lens 410, 430, respectively, is inserted into thetop of the respective aperture, seated in a respective seat 618-2 andaffixed thereto, if desired. Referring to FIG. 35C, thereafter, the toplenslet 410A, 430A, of each lens 410, 430, respectively, is insertedinto the top of the respective aperture, seated in a respective seat618-1 and affixed thereto, if desired. In some embodiments, the toplenslet and the middle lenslet are built into one assembly, and insertedtogether.

In this particularly embodiment, it may be advantageous to insert thebottom lenslet through a bottom portion of the aperture because the stopsurface for the bottom lenslet faces toward the bottom of the aperture.Similarly, it may be advantageous to insert the top lenslet and themiddle lenslet through a top portion of the aperture because the stopsurface for the top lenslet and the stop surface for the middle lensleteach face toward the top portion of the aperture.

It should be understood however, that any suitable configuration may beemployed. In some embodiments, for example, the stop surface for themiddle lenslet may face toward the bottom portion of the aperture, suchthat the middle lenslet may be inserted into the support portion throughthe bottom portion of the aperture, e.g., prior to inserting the bottomlenslet into the support. In some other embodiments, each of the stopsurfaces may face in one direction, such that all of the lenslets areinserted through the same portion of the aperture.

In some embodiments, the lens and support are manufactured as a singlemolded component. In some embodiments the lens may be manufactured withtabs that may be used to form the support.

In some embodiments, the support 320 is coupled and/or affixed directlyor indirectly, to the image device. For example, the support 320 may bedirectly coupled and affixed to the image device (e.g., using adhesive)or indirectly coupled and/or or affixed to the image device via anintermediate support member (not shown).

The x and y dimensions of the support 320 may be, for example,approximately the same (in one or more dimensions) as the image device,approximately the same (in one or more dimensions) as the arrangement ofthe optics portions 330A-330D and/or approximately the same (in one ormore dimensions) as the arrangement of the sensor arrays 310A-310D. Oneadvantage of such dimensioning is that it helps keep the x and ydimensions of the digital camera apparatus as small as possible.

In some embodiments the support may have dimensions similar to one ormore of the dimensions of the embodiment of the support shown in FIGS.28A-28D.

FIG. 36 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of the digital camera apparatus and printedcircuit board shown in FIG. 32 except that this embodiment employs thesupport 320 and lens elements 410, 430 shown in FIGS. 35A-35C.

In some embodiments, the digital camera apparatus 300 has dimensions ofabout 2.5 mm×6 mm×6 mm. For example, the thickness may be equal to about2.5 mm, the length may be equal to about 6 mm and the width may be equalto about 6 mm. In some of such embodiments, the digital camera apparatushas one or more sensor arrays having a total of 1.3 M pixels, althoughother configurations may be employed (e.g., different thickness, width,length and number of pixels).

The digital camera apparatus 300 may be assembled and mounted to theprinted circuit board in any manner. In some embodiments, the digitalcamera apparatus is assembled and mounted to the printed circuit board720 in a manner that is similar to that set forth above for theembodiment of the digital camera apparatus 300 and printed circuit boardshown 720 in FIGS. 33A-33F, except that the bottom lenslets 410C, 430C,may be seated in and affixed to the support, if desired, prior topositioning the support on the second bond layer. The middle and toplenslets of the lenses, respectively, may be seated in and affixed tothe support, if desired, after the support is positioned on the secondbond layer 724.

FIG. 37, is a schematic cross section view of an alternative support 320that may be employed to support the lenses 410, 430 of FIGS. 11A-11B,13A-13B and to position such lenses in registration with a respectivesensor array, at least in part, in accordance with another embodiment ofthe present invention. The support 320 in this embodiment is similar tothe embodiment of the support 320 shown in FIG. 34 except that thesupport 320 in this embodiment defines outer walls 760A-760D that arewider than outer walls 760A-760D defined by the embodiment of thesupport shown in FIG. 34.

Each optics portion, e.g., optics portions 330A-330S, may be assembledin and secured in the respective seats in any suitable manner, forexample, but not limited to, in the manner set forth above with respectto the embodiment of the support and optics portions shown in FIGS.35A-35C.

FIG. 38, is a schematic cross section view of an alternative support 320that may be employed to support the lenses 410, 430 of FIGS. 11A-11B,13A-13B and to position such lenses in registration with a respectivesensor array, at least in part, in accordance with another embodiment ofthe present invention. The support 320 in this embodiment is similar tothe embodiment of the support 320 shown in FIG. 34 except that thesupport 320 in this embodiment defines outer and inner walls 760A-760Dthat are wider than outer and inner walls 760A-760D defined by theembodiment of the support 320 shown in FIG. 34.

Each optics portion may be assembled in and secured in the respectiveseats in any suitable manner, for example, but not limited to, in themanner set forth above with respect to the embodiment of the support andoptics portions shown in FIGS. 35A-35C.

FIG. 39 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of the digital camera apparatus 300 andprinted circuit board 720 shown in FIG. 36 except that this embodimentemploys the support 320 and lens elements 410, 430 shown in FIG. 37.

The digital camera apparatus may be assembled and mounted to the printedcircuit board in any manner. In some embodiments, for example, thedigital camera apparatus is assembled and mounted to the printed circuitboard in a manner that is similar to that set forth for the embodimentof the digital camera apparatus and printed circuit board shown in FIG.36.

FIG. 40 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of the digital camera apparatus 300 andprinted circuit board shown in FIG. 36 except that this embodimentemploys the support 320 and lens elements 410, 430 shown in FIG. 38.

The digital camera apparatus 300 may be assembled and mounted to theprinted circuit board 720 in any manner. In some embodiments, forexample, the digital camera apparatus 300 is assembled and mounted tothe printed circuit board 720 in a manner that is similar to that setforth for the embodiment of the digital camera apparatus 300 and printedcircuit board 720 shown in FIG. 36.

FIGS. 41A-41D, are schematic cross sectional views of seatingconfigurations 770-776 that may be used in supporting and positioningthe lenses of FIGS. 15A-15B, 15D-15E, respectively, in accordance withfurther embodiments.

In the seating configuration shown in FIG. 41A, the top lenslet 450A,middle lenslet 450B and bottom lenslet 450C are each inserted through abottom portion of an aperture (or through a top portion of an aperture)one at a time, as an assembly, or a combination thereof.

In the seating configuration of FIG. 41B, the top lenslet 452A, middlelenslet 452B and bottom lenslet 452C are each inserted through a topportion of an aperture (or through a bottom portion of an aperture), oneat a time, as an assembly, or a combination thereof.

In the seating configuration of FIG. 41C, the top lenslet 456A may beinserted, for example, through a top portion of an aperture. The middlelenslet 456B and the bottom lenslet 456C may be inserted, for example,through a bottom portion of an aperture, one at a time, oralternatively, the middle lenslet and the bottom lenslet may be builtinto one assembly, and inserted together.

In the seating configuration of FIG. 41D, the middle lenslet 458B andthe top lenslet 458A are inserted through a top portion of an aperture,one at a time, or alternatively, the middle lenslet 458B and the toplenslet 458A may be built into one assembly, and inserted together. Thebottom lenslet 458C is inserted through a bottom portion of an aperture.

As with each of the embodiments disclosed herein, these embodiments maybe employed alone or in combination with one or more of the otherembodiments (or portions thereof) disclosed and illustrated herein.

In that regard, FIGS. 42-44 are schematic cross section views ofsupports 32 that employ the seating configurations of FIGS. 41B-41D,respectively, to support the lens 452A-452C, 456A-456C, 458A-458C, shownin FIGS. 15B-15D, respectively, and to position such lens inregistration with a respective sensor array, at least in part, inaccordance with further embodiments.

In that regard, FIGS. 42-44 are schematic cross section views ofsupports that employ the seating configurations of FIGS. 41B-41D,respectively, to support the lens 452A-452C shown in FIGS. 15B-15D,respectively, and to position such lens in registration with arespective sensor array, at least in part, in accordance with furtherembodiments.

FIG. 45 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of digital camera apparatus 300 and theprinted circuit board shown in FIG. 36 except this embodiment employssupport 320 and lens elements shown in FIG. 42.

The digital camera apparatus 300 may be assembled and mounted to theprinted circuit board in any manner. In some embodiments, for example,the digital camera apparatus is assembled 300 and mounted to the printedcircuit board 720 in a manner that is similar to that set forth for theembodiment of the digital camera apparatus and printed circuit boardshown in FIG. 36, although it may be advantageous to assemble thelenslets into the support using a manner similar to the manner set forthabove for the seating configuration shown in FIG. 41B.

FIG. 46 is a schematic cross-sectional view of digital camera apparatus300 and printed circuit board 720 of a digital camera on which digitalcamera apparatus 300 may be mounted, in accordance with anotherembodiment of the present invention. This embodiment is similar to theembodiment of the digital camera apparatus and printed circuit boardshown in FIG. 36 except this embodiment employs the support and lenselements shown in FIG. 43.

The digital camera apparatus 300 may be assembled and mounted to theprinted circuit board 720 in any manner. In some embodiments, forexample, the digital camera apparatus 300 is assembled and mounted tothe printed circuit board in a manner that is similar to that set forthfor the embodiment of the digital camera apparatus and printed circuitboard shown in FIG. 36, although it may be advantageous to assemble thelenslets into the support using a manner similar to the manner set forthabove for the seating configuration shown in FIG. 41C.

FIG. 47 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 720 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of the digital camera apparatus 300 andprinted circuit board 720 shown in FIG. 36 except that this embodimentemploys the support and lens elements shown in FIG. 44.

The digital camera apparatus 300 may be assembled and mounted to theprinted circuit board 720 in any manner. In some embodiments, forexample, the digital camera apparatus 300 is assembled and mounted tothe printed circuit board 720 in a manner that is similar to that setforth for the embodiment of the digital camera apparatus and printedcircuit board shown in FIG. 36, although it may be advantageous toassemble the lenslets into the support using a manner similar to themanner set forth above for the seating configuration shown in FIG. 41D.

In some embodiments, the digital camera apparatus 300 includes one ormore additional structures and/or devices, for example, but not limitedto, one or more additional integrated circuits, one or more outputdevices and/or one or more input devices. The one or more output devicesmay include any type or types of output devices, for example, but notlimited to one or more display devices, one or more speakers and/or anycombination thereof. The one or more input devices may include any typeor types of input devices, for example, but not limited to one or moremicrophones. The additional structures and/or devices may be disposed,in any suitable location, for example, but not limited to, adjacent tothe image device.

The additional structures and/or devices may comprise any type ofmaterial(s) and may have any configuration and/or construction. In someembodiments, for example, the additional structures and/or devices maycomprise silicon, semiconductor, glass, ceramic, plastic, or metallicmaterials and/or a combination thereof. The one or more additionalstructures and/or devices may be fabricated separate from one another,integral with one another and/or any combination thereof. The one ormore additional structures and/or devices may be fabricated separatefrom the camera channels, integral with the camera channels and/or anycombination thereof. The one or more additional structures and/ordevices may or may not be physically connected to the processor, the oneor more camera channels or any portions thereof. The one or moreadditional structures and/or devices may or may not be electricallyconnected to the processor and/or one or more camera channels orportions thereof.

FIG. 48 is a schematic representation of a digital camera apparatus 300that includes a second device 780 in accordance with another embodimentof the present invention. The second device 780 may comprise, forexample, but is not limited to, an integrated circuit including any typeof circuit or circuits, for example, but not limited to, one or moreportions of the processor, one or more portions of a memory or anadditional memory, one or more portions of the processor (e.g., one ormore portions of a post processor) and/or any other types of circuits.

For example, in some embodiments, the digital camera apparatus 300includes a memory section that is supplied with and/or stores one, someor all of the images and/or other information generated or used by thedigital camera apparatus and/or or any other information from any sourceand desired to be stored for any duration. The memory section may supplyone or more of such images and/or such other information to one or moreother devices and/or to one or more portions of the processor, forexample, to be further processed and/or to be supplied to one or moreother devices. The memory section may be integrated into or disposed on(for example, as a discrete component) the same or different substrateas one some or all of the sensor arrays. The memory section may be, forexample, part of or integrated into the processor (which may beintegrated into or disposed on (for example, as a discrete component)the same or different substrate as one some or all of the sensor arrays)and/or coupled to one or more portions of the processor via one or morecommunication links. In some embodiments, the memory section is alsocoupled to one or more other devices via one or more communicationlinks. In such embodiments, the memory section may supply one or more ofthe stored images and/or other information to one or more of the one ormore other devices, directly (i.e., without passing through the anyother portion of the processor) via one or more of the one or morecommunication links, although this is not required.

The second device 780 may be disposed in any suitable location orlocations. However, in some embodiments, the second device 780 isdisposed generally adjacent to or in the vicinity of an associated imagedevice, e.g., image device 520, or associated processor.

The circuit or circuits of the second device 780 may be connected, e.g.,via one or more communication links, to one or more portions of theprocessor 340, one or more of the camera channels, one or more otherdevices and/or any combination thereof. In some embodiments, the one ormore communication links include one or more pads on the image deviceand the second device 780 and one or more electrical connectors havingone or more electrically conductive members connecting one or more ofthe pads on the image device to one or more of the pads on the seconddevice. In some embodiments, the one or more communication links includeone or more bump bonds that electrically connect one or more circuits onthe image device to one or more circuits on the second device.

The second device 780 may have any size and shape and may or may nothave the same configuration as the image device. In some embodiments,the second device 780 has a length and a width that are less than orequal to the length and width, respectively of the optical assembly, thesensor subassembly and/or the image device. In some other embodiments,the second device 780 has a length or a width that is greater than thelength or width, respectively of the optical assembly, the sensorsubassembly and/or the image device.

Although the processor is shown separate from the image device andsecond device, it should be understood that the processor may have anyconfiguration and that the processor, or portions thereof, may bedisposed in any location or locations. In some embodiments, one, some orall portions of the processor are disposed on or integrated into theimage device. In some embodiments one, some or all portions of theprocessor are disposed on or integrated into the second device. In someof such embodiments one or more portions of the processor are disposedon the image device and one or more portions of the processor aredisposed on or integrated into the second device. For example, certainoperations of the processor may be distributed to or performed bycircuitry that is integrated in or disposed on the same substrate orsubstrates as one or more of the one or more of the sensor arrays andcertain operations of the processor are distributed to or performed bycircuitry that is integrated in or disposed on one or more substratesthat are different from (whether such one or more different substratesare physically located within the camera or not) the substrates the oneor more of the sensor arrays are integrated in or disposed on.

In some embodiments, the digital camera apparatus may further includeone or more addition integrated circuits devices, e.g., a thirdintegrated circuit device (not shown). The one or more additionalintegrated circuits device may have any size and shape and may or maynot have the same configuration as one another, the image device or thesecond device. In some embodiments, the third integrated circuit devicehas a length and a width that are equal to the length and width,respectively of the optical assembly, the sensor subassembly and/or theimage device. In some other embodiments, the third integrated circuitdevice has a length or a width that is greater than or less than thelength or width, respectively of the optical subassembly, the sensorsubassembly and/or the image device.

FIG. 49 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus may be mounted, in accordance withanother embodiment of the present invention. This embodiment is similarto the embodiment of the digital camera apparatus and printed circuitboard shown in FIG. 36 except that this embodiment includes a seconddevice 780 such as, for example, as shown in FIG. 48. The second device780 is disposed superjacent a third bond layer 782, which is disposedsuperjacent the printed circuit board.

The third bond layer 782 may comprise any suitable material(s), forexample, but not limited to adhesive, and may comprise any suitableconfiguration. The third bond layer 782 may comprise the samematerial(s) as the first and/or second bond layers 722, 724, althoughthis is not required.

In some embodiments, the digital camera apparatus 300 has dimensions ofabout 2.5 mm×6 mm×6 mm. For example, the thickness may be equal to about2.5 mm, the length may be equal to about 6 mm and the width may be equalto about 6 mm. In some of such embodiments, the digital camera apparatushas one or more sensor arrays having a total of 1.3 M pixels, althoughother configurations may be employed (e.g., different thickness, width,length and number of pixels).

The digital camera apparatus 300 may be assembled and/or mounted in anymanner. FIGS. 50A-50F shows one such embodiment for assembling andmounting the digital camera apparatus. Referring to FIG. 50A, initially,the second device is provided. Referring to FIG. 50B, a bond layer 724is provided on one or more regions of one or more surfaces of the seconddevice. Such regions define one or more mounting regions for the imagedevice. Referring to FIG. 50C, the image device 520 is thereafterpositioned on the bond layer 724. In some embodiments, force may beapplied to help drive any trapped air out from between the image device520 and second device 780. In some embodiments, heat and/or force may beapplied to provide conditions to activate and/or cure the bond layer toform a bond between the image device and the second device. Referring toFIG. 50D, a bond layer 722 is provided on one or more regions of one ormore surfaces of the image device 520. Such regions define one or moremounting regions for the support 320. Referring to FIG. 50E, the support320 is thereafter positioned on the bond layer 722. In some embodiments,force may be applied to help drive any trapped air out from between theimage device 520 and support 320. In some embodiments, heat and/or forcemay be applied to provide conditions to activate and/or cure the bondlayer to form a bond between the image device and the support. Referringto FIG. 50F, one or more optics portions, e.g., optics portions330A-330D may thereafter be seated in and/or affixed to the support 320.Referring to FIG. 50G, a bond layer 782 is provided on one or moreregions of one or more surfaces of the printed circuit board 720. Suchregions define one or more mounting regions for the digital cameraapparatus 300. Referring to FIG. 50H, the digital camera apparatus isthereafter positioned on the bond layer 782. One or more electricalconductors 750 may be installed to connect one or more of the pads 742on the image device 520 to one or more pads on the circuit board 732.One or more electrical conductors 790 may be installed to connect one ormore of the pads on the image device 792 to one or more pads on thesecond device 794.

FIG. 51 is a schematic representation of an exemplary digital cameraapparatus, digital camera apparatus 300 includes a spacer 800 disposedbetween the support 320 and the image device 520, in accordance withanother embodiment of the present invention. In some embodiments, thespacer 800 helps position the optics portions, e.g., optics portions330A-330D, at the respective desired distance or distances from therespective sensor arrays, e.g., 310A-310D, respectively. In thisembodiment, the spacer 800 extend in an axial direction and defineswalls 802, which define apertures, e.g., apertures 804-804D (e.g., forcamera channels 350A-350D, respectively), for transmission of light, tohelp limit, minimize and/or eliminate light “cross talk” between thecamera channels and to help limit, minimize and/or eliminate “entry” oflight from outside the digital camera apparatus.

The spacer 800 may comprise any type of material(s) and may have anyconfiguration and/or construction. In some embodiments, for example, thespacer 800 comprises silicon, semiconductor, glass, ceramic, plastic, ormetallic materials and/or a combination thereof. If the spacer has morethan one portion, such portions may be fabricated separate from oneanother, integral with one another and/or any combination thereof.

The spacer 800 may be fabricated separately from and/or integral withthe support 320 or support portions 600A-600D.

The spacer 800 may be a solid device that may offer a wide range ofoptions for manufacturing and material, however other forms of devicesmay also be employed. In some embodiments, for example, the spacercomprises a plate (e.g., a thin plate) that defines the walls andapertures of the spacer. Apertures, e.g. apertures 804A-804D, may beformed by machining (e.g., boring) or any other suitable manner. In someembodiments, the spacer is fabricated as a casting with the aperturesdefined therein (e.g., using a mold with projections that define theapertures and seats of the one or more support portions of the spacer).

Although the processor is shown separate from the image device, itshould be understood that the processor may have any configuration andthat the processor, or portions thereof, may be disposed in any locationor locations. In some embodiments, one, some or all portions of theprocessor are disposed on the image device.

As with each of the embodiments of the present invention, thisembodiment may be employed alone or in combination with one or more ofthe other embodiments disclosed herein, or portions thereof.

FIG. 52 is a schematic representation of a digital camera apparatus 300that includes a spacer 800 disposed between the support 320 and theimage device 520, in accordance with another embodiment of the presentinvention. This embodiment of the spacer 800 is similar to theembodiment of the spacer 500 shown in FIG. 51 except that the spacer 800in this embodiment defines only one aperture 804 for transmission oflight and may not help limit, minimize and/or eliminate light “crosstalk” between the camera channels, e.g., camera channels 350A-350D.

FIG. 53 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus 300 may be mounted, in accordancewith another embodiment of the present invention. This embodiment issimilar to the embodiment of the digital camera apparatus and printedcircuit board shown in FIG. 36 except that this embodiment includes aspacer 800 such as, for example, as shown in FIG. 51. The spacer 800 isdisposed superjacent a bond layer 782, which is disposed superjacent theimage device 520.

The bond layer 782 may comprise any suitable material(s), for example,but not limited to adhesive, and may comprise any suitableconfiguration. The bond layer may comprise the same material(s) as otherbond layers although this is not required.

In some embodiments, the digital camera apparatus has dimensions ofabout 2.5 mm×6 mm×6 mm. For example, the thickness may be equal to about2.5 mm, the length may be equal to about 6 mm and the width may be equalto about 6 mm. In some of such embodiments, the digital camera apparatushas one or more sensor arrays having a total of 1.3 M pixels, althoughother configurations may be employed (e.g., different thickness, width,length and number of pixels).

The digital camera apparatus may be assembled and mounted in any manner.FIGS. 54A-54F shows one such embodiment for assembling and mounting thedigital camera apparatus 300. Referring to FIG. 54A, initially, theimage device 520 is provided. Referring to FIG. 54B, a bond layer 782 isprovided on one or more regions of one or more surfaces of the imagedevice. Such regions define one or more mounting regions for the spacer800. Referring to FIG. 54C, the spacer 800 is thereafter positioned onthe bond layer 782. In some embodiments, force may be applied to helpdrive any trapped air out from between the spacer 800 and the imagedevice 520. In some embodiments, heat and/or force may be applied toprovide conditions to activate and/or cure the bond layer to form a bondbetween the spacer and the image device. Referring to FIGS. 54D-54E, abond layer 722 is provided on one or more regions of one or moresurfaces of the spacer 800. Such regions define one or more mountingregions for the one or more support portions of the support 320, whichis thereafter positioned on the bond layer 722. In some embodiments,force may be applied to help drive any trapped air out from between thespacer 800 and the one or more support portions of the support 320. Insome embodiments, heat and/or force may be applied to provide conditionsto activate and/or cure the bond layer to form a bond between the spacerand the one or more support portions of the support. Referring to FIG.54F, one or more optics portions, e.g., optics portions 330A-330D maythereafter be seated in and/or affixed to the support 320. Referring toFIG. 54G, a bond layer 724 is provided on one or more regions of one ormore surfaces of the printed circuit board 720. Such regions define oneor more mounting regions for the digital camera apparatus 300. Referringto FIG. 54H, the digital camera apparatus is thereafter positioned onthe bond layer 724. One or more electrical conductors 750 may beinstalled to connect one or more of the pads 742 on the image device toone or more pads 732 on the circuit board.

As with each of the embodiments of the present invention, thisembodiment may be employed alone or in combination with one or more ofthe other embodiments disclosed herein, or portions thereof.

For example, FIG. 55 is a schematic representation of a digital cameraapparatus 300 that includes a second device and a spacer 800, inaccordance with another embodiment of the present invention.

Although the processor is shown separate from the image device andsecond device, it should be understood that the processor 340 may haveany configuration and that the processor, or portions thereof, may bedisposed in any location or locations. In some embodiments, one, some orall portions of the processor are disposed on the image device. In someembodiments one, some or all portions of the processor are disposed onthe second device. In some of such embodiments one or more portions ofthe processor are disposed on the image device and one or more portionsof the processor are disposed on the second device.

FIG. 56 is a schematic cross-sectional view of a digital cameraapparatus 300 and a printed circuit board 720 of a digital camera onwhich the digital camera apparatus may be mounted, in accordance withanother embodiment of the present invention. This embodiment is similarto the embodiment of the digital camera apparatus and printed circuitboard shown in FIG. 53 except that this embodiment includes a seconddevice 780. The second device 800 is disposed superjacent a bond layer808, which is disposed superjacent printed circuit board 720.

The bond layer 808 may comprise any suitable material(s), for example,but not limited to adhesive, and may comprise any suitableconfiguration. The bond layer may comprise the same material(s) as otherbond layers although this is not required.

In some embodiments, the digital camera apparatus has dimensions ofabout 2.5 mm×6 mm×6 mm. For example, the thickness may be equal to about2.5 mm, the length may be equal to about 6 mm and the width may be equalto about 6 mm. In some of such embodiments, the digital camera apparatushas one or more sensor arrays having a total of 1.3 M pixels, althoughother configurations may be employed (e.g., different thickness, width,length and number of pixels).

The digital camera apparatus 300 may be assembled and mounted in anymanner. FIGS. 57A-57F shows one such embodiment for assembling andmounting the digital camera apparatus. Referring to FIG. 57A, initially,the second device 780 is provided. Referring to FIG. 57B, a bond layer724 is provided on one or more regions of one or more surfaces of thesecond device 780. Such regions define one or more mounting regions forthe image device 520. Referring to FIG. 57C, the image device isthereafter positioned on the bond layer 724. In some embodiments, forcemay be applied to help drive any trapped air out from between the imagedevice and second device 780. In some embodiments, heat and/or force maybe applied to provide conditions to activate and/or cure the bond layerto form a bond between the image device and the second device 780.Referring to FIG. 57D, a bond layer 782 is provided on one or moreregions of one or more surfaces of the image device. Such regions defineone or more mounting regions for the spacer 800. Referring to FIG. 57E,the spacer 800 is thereafter positioned on the bond layer 782. In someembodiments, force may be applied to help drive any trapped air out frombetween the spacer and the image device. In some embodiments, heatand/or force may be applied to provide conditions to activate and/orcure the bond layer to form a bond between the spacer and the imagedevice. Referring to FIGS. 54E-54G, a bond layer 722 is provided on oneor more regions of one or more surfaces of the spacer 800. Such regionsdefine one or more mounting regions for the one or more support portionsof the support 320, which is thereafter positioned on the bond layer722. In some embodiments, force may be applied to help drive any trappedair out from between the spacer and the one or more support portions ofthe support 320. In some embodiments, heat and/or force may be appliedto provide conditions to activate and/or cure the bond layer to form abond between the spacer 800 and the one or more support portions of thesupport 320. One or more optics portions, e.g., optics portions330A-330D may thereafter be seated in and/or affixed to the support.Referring to FIG. 57G, a bond layer 808 is provided on one or moreregions of one or more surfaces of the printed circuit board 720. Suchregions define one or more mounting regions for the digital cameraapparatus 300. Referring to FIG. 57H, the digital camera apparatus isthereafter positioned on the bond layer 782. One or more electricalconductors 750 may be installed to connect one or more of the pads 742on the image device to one or more pads on the circuit board 732. One ormore electrical conductors 790 may be installed to connect one or moreof the pads 742 on the image device to one or more pads on the seconddevice 780.

As stated above, each of the embodiments disclosed herein may beemployed alone or in combination with one or more of the otherembodiments disclosed herein, or portions thereof.

For example, in some embodiments, one or more of the supports shown inFIGS. 37-38 and 42-44 are employed in one or more of the embodiments ofthe digital camera apparatus shown in FIGS. 48-57.

For example, FIGS. 58-62 are schematic cross-sectional views of adigital camera apparatus and a printed circuit board of a digital cameraon which the digital camera apparatus may be mounted, in accordance withfurther embodiments of the present invention. These embodiments aresimilar to the embodiment of the digital camera apparatus and printedcircuit board shown in FIG. 49 except that the support and the opticsportions have configurations similar to the support and optics portionsshown in FIGS. 37-38, 42-44, respectively.

FIGS. 63-67 are schematic cross-sectional views of a digital cameraapparatus and a printed circuit board of a digital camera on which thedigital camera apparatus may be mounted, in accordance with furtherembodiments of the present invention. These embodiments are similar tothe embodiment of the digital camera apparatus and printed circuit boardshown in FIG. 53 except that the support and the optics portions haveconfigurations similar to the support and optics portions shown in FIGS.37-38, 42-44, respectively.

In some embodiments herein, one or more electrical or electro mechanicaldevices are disposed in or on a support and/or spacer. In some of suchembodiment, one or more electrical conductors may connect one or more ofsuch devices to one or more circuits on the image device and/or anotherdevice, for example, to provide power, control signals and/or datasignals to and/or from one or more of such device. One or more of suchelectrical conductors may be in the form of an electrical connector,although this is not required. The electrical conductors may be disposedon one or more outside surface and/or may extend through one or moreportion of one or more portions of the digital camera apparatus, e.g., asupport, a spacer, an image device, if present, or combinations thereof.For example, in some embodiments, one or more electrical conductors,e.g., conductors 810, 812 (FIGS. 63-72) are provided and extend over oneor more surfaces of, or through one or more portions of, a support(e.g., over or through one or more support portions, e.g., 600A-600D)and/or spacer (e.g., over or through one or more walls, e.g., walls 602)so as to connect to one or more circuits on or in the image device oranother device.

FIGS. 68-72 are schematic cross-sectional views of a digital camerasubsystem and a printed circuit board of a digital camera on which thedigital camera apparatus may be mounted, in accordance with furtherembodiments of the present invention. These embodiments are similar tothe embodiment of the digital camera apparatus and printed circuit boardshown in FIG. 56 except that the support and the optics portions haveconfigurations similar to the support and optics portions shown in FIGS.37-38, 42-44, respectively.

FIGS. 73A-73B are schematic elevational and cross sectional views,respectively, of a support in accordance with another embodiment of thepresent invention. In this embodiment, one or more of the supportportions are spaced apart from one another and/or isolated from oneanother, e.g., by on or more clearances or spaces, e.g., clearance 816.

FIG. 74 is a schematic cross sectional view of a support in accordancewith another embodiment of the present invention. In this embodiment,the support includes one or more support portions disposed superjacent(e.g., on or above) one or more other support portions. In some suchembodiments, the support portions may be spaced apart from one anotherand/or isolated from one another in the z direction, e.g., by clearancesor spaces, e.g., clearance 816.

As stated above, it should be understood that each of the embodimentsdescribed above may be used alone or in combination with any otherembodiment(s) or portion thereof described herein or known to those ofordinary skill in the art.

For example, in some embodiments the support is adapted to receive oneor more optics portions of a first size and shape and one or more opticsportions of a second size and shape that is different than the firstsize and/or first shape. In some embodiments, optics portions of furthersizes and shapes may also be received, e.g., a third size and shape, afourth size and shape, a fifth size and shape, etc.

Referring to FIG. 75, in some embodiments, one or more of the supportsdisclosed herein is provided with one or more curved portions, e.g.,curved portions 818A-818D. Such aspect may be advantageous, for example,in some embodiments in which it is desired to reduce and/or minimize thedimensions of the digital camera apparatus.

FIGS. 76A-76C are schematic views of a digital camera apparatus thatincludes one or more output devices 820 in accordance with anotherembodiment of the present invention. FIG. 76A is a schematic perspectiveview of one embodiment of a digital camera apparatus that includes oneor more output devices. FIGS. 76B-76C are schematic front and backperspective views, respectively, of an output device 820 in accordancewith one embodiment of the present invention.

In some embodiments, the one or more output devices 820 are in the formof one or more display devices, however, other types of output devicesmay also be employed. In some embodiments, the one or more displaydevices are in the form of one or more micro displays.

The one or more display devices may be disposed in any suitable locationor locations. In some embodiments, it may be advantageous to collectlight (for the one or more camera channels) on one side of the digitalcamera assembly and to provide one or more of the one or more outputdisplays, e.g., output display 820, on the other side of the digitalcamera assembly. In the illustrated embodiment, the digital cameraapparatus has first and second sides generally opposite one another. Theone or more camera channels are positioned to receive light through afirst side of the digital camera apparatus. One or more of the displaydevices are positioned to transmit light (e.g., one or more displayimages) from the second side of the digital camera apparatus. In someembodiments, such a configuration may make it possible to provide adigital camera apparatus that is very thin (e.g., in the z direction).Other configurations may also be employed. In some embodiments, one ormore of the display devices is positioned generally adjacent to theimage device, although this is not required.

The one or more display devices may be connected to the processor, oneor more of the camera channels or any combination thereof, via one ormore communication links. In some embodiments, the one or morecommunication links include one or more pads on the image device and theone or more display devices and one or more electrical connectors havingone or more electrically conductive members connecting one or more ofthe pads on the image device to one or more of the pads on the one ormore display devices. In some embodiments, the one or more communicationlinks include one or more bump bonds that electrically connect one ormore circuits on the image device to one or more circuits on the one ormore display devices.

The one or more display devices may have any size and shape and may ormay not have the same configuration as one another (e.g., type, size,shape, resolution). In some embodiments, one or more of the one or moredisplay devices has a length and a width that are less than or equal tothe length and width, respectively of the optical assembly, the sensorsubassembly and/or the image device. In some embodiments, one or more ofthe one or more display devices has a length or a width that is greaterthan the length or width, respectively of the optical assembly, thesensor subassembly and/or the image device. In some embodiments, each ofthe camera channels is connected to its own display device. In someother embodiments, two or more camera channels, e.g., camera channels350A-350B, are connected to a first display and one or more of the othercamera channels, e.g., camera channel 350C-350D, is connected to asecond display. In some embodiments, one of the one or more displays isconnected to the processor to display a combined image based at least inpart on images from each of the camera channels.

As with each of the embodiments disclosed herein, the above embodimentsmay be employed alone or in combination with one or more of the otherembodiments disclosed herein, or portions thereof.

Thus, in some embodiments, the digital camera apparatus 300 furtherinclude a spacer 800 (see for example, FIG. 76D) and/or one or moreelectrical conductors 822 to connect one or more circuits of the one ormore output devices, e.g., output device 820, to one or more circuits inone or more other portions of the subsystem 300, one or more circuits ofa processor 340 that may be disposed on the image device 520 (see forexample, FIG. 76E).

Further, in some embodiments, the digital camera apparatus may furtherinclude one or more illumination devices and/or a support having one ormore actuators (such a support may comprise, for example, a frame havingone or more actuators) (e.g., MEMS actuators, for example, comb typeMEMS actuators) to move one or more of the optics portions of the camerachannels. In some of the latter embodiments, the digital cameraapparatus includes one or more illumination devices and/or a supporthaving one or more actuators (such a support may comprise, for example,a frame having one or more actuators) (e.g., MEMS actuators).

FIGS. 77A-77C are schematic views of a digital camera apparatus thatincludes one or more input devices 830 in accordance with anotherembodiment of the present invention. More particularly, FIG. 77A is aschematic perspective view of one embodiment of a digital cameraapparatus that includes one or more input devices. FIGS. 77B-77C areenlarged schematic front and back perspective views, respectively, of aninput device in accordance with one embodiment of the present invention.In this embodiment, the one or more input devices are in the form of oneor more audio input devices, e.g., one or more microphones, however,other types of input devices may also be employed. In some embodiments,the one or more microphones are in the form of one or more siliconmicrophones.

The one or more audio input devices may be disposed in any suitablelocation or locations. In some embodiments, it may be advantageous tocollect light (for the one or more camera channels) on one side of thedigital camera assembly and to collect sound from the same side of thedigital camera subassembly. In the illustrated embodiment, the digitalcamera apparatus has first and second sides generally opposite oneanother. The one or more camera channels are positioned to receive lightthrough a first side of the digital camera apparatus. One or more of theaudio input devices may be positioned to receive audio input (e.g.,sound) from the first side of the digital camera apparatus. In someembodiments, such a configuration may make it possible to provide adigital camera apparatus that is very thin (e.g., in the z direction).Other configurations may also be employed. In some embodiments, one ormore of the audio input devices is disposed on and/or integral with oneor more portions of the support, although this is not required.

The one or more audio input devices may be connected to the processorvia one or more communication links. In some embodiments, the one ormore communication links include one or more pads on the image deviceand the one or more audio input devices and one or more electricalconnectors having one or more electrically conductive members connectingone or more of the pads on the image device to one or more of the padson the audio input devices. In some embodiments, the one or morecommunication links include one or more bump bonds that electricallyconnect one or more circuits on the image device to one or more circuitson the one or more audio input devices.

The one or more audio input devices may have any size and shape and mayor may not have the same configuration as one another (e.g., type, size,shape, resolution). In some embodiments, one or more of the one or moreaudio input devices has a length and a width that are less than or equalto the length and width, respectively of the optical assembly, thesensor subassembly and/or the image device. In some embodiments, one ormore of the one or more audio input devices has a length or a width thatis greater than the length or width, respectively of the opticalassembly, the sensor subassembly and/or the image device.

FIGS. 77G-77L are schematic perspective view of digital camera apparatusin accordance with further embodiments. Such embodiments have inputdevices with configurations and/or in arrangements that are differentthan the configuration and/or arrangement of the input device shown inFIG. 77A. Other configurations and/or arrangements may also be employed.

As with each of the embodiments disclosed herein, this embodiment of thepresent invention may be employed alone or in combination with one ormore of the other embodiments disclosed herein, or portion thereof.

Thus, in some embodiments, the digital camera apparatus 300 furtherinclude a spacer 800, one or more electrical conductors 822 to connectone or more circuits of the one or more input devices, e.g., inputdevice 830, to one or more circuits in one or more other portions of thesubsystem 300, one or more circuits of a processor 340 that may bedisposed on the image device 520 and/or one or more additional devices,e.g., one or more output devices 820.

For example, FIG. 77D is a schematic perspective view of one embodimentof a digital camera apparatus that includes an input device and a spacer800. FIG. 77E is a schematic perspective view of one embodiment of adigital camera apparatus 300 that includes spacer 800 and one or moreadditional devices, e.g., one or more output devices 820. The imagedevice is shown with one or more pads connected to one or more circuitsdisposed on or in the image device. FIG. 77F is a schematic perspectiveview of one embodiment of a digital camera apparatus that includes aninput device, a spacer and an additional device (e.g., a display and/ora second integrated circuit device adjacent to the image device). Theimage device is shown with one or more pads connected to one or morecircuits disposed on or in the image device.

Moreover, in some embodiments, the digital camera apparatus may furtherinclude a support having one or more actuators (such a support maycomprise, for example, a frame having one or more actuators) (e.g., MEMSactuators, for example, comb type MEMS actuators) to move one or more ofthe optics portions of the camera channels, one or more display devicesand/or one or more illumination devices (e.g., one or more lightemitting diodes (LEDs) with high output intensity. In some of the latterembodiments, the digital camera apparatus includes one or more audioinput devices, a support having one or more actuators (such a supportmay comprise, for example, a frame having one or more actuators) (e.g.,MEMS actuators, for example, comb type MEMS actuators) to move one ormore of the optics portions of the camera channels, one or more displaydevices and one or more illumination devices.

Figure is a schematic representation of a digital camera apparatus thatincludes one or more audio input devices, one or more display devicesand one or more illumination devices.

The digital camera apparatus may be assembled and/or mounted in anymanner, for example, but not limited to in a manner similar to thatemployed in one or more of the embodiments disclosed herein.

Any of the embodiments of the present invention may include one or moreillumination units to improve and/or enhance image acquisition by theone or more camera channels (and, in particular, the one or more sensorarrays), facilitate range detection to an object, shape detection of anobject, and covert imaging (i.e., imaging that is not observable to thehuman eye).

FIGS. 78A-78B are schematic block diagrams of digital camera apparatushaving one or more illumination units, e.g., illumination units 840, inaccordance with further embodiments of the present invention. Theillumination units may provide passive (for example, no illumination),active (for example, constant illumination), constant and/or gatedactive illumination (for example, pulsed illumination that ispredetermined, preset or processor controlled, and/or pulsedillumination that is user/operator programmable). The one or moreillumination units may be disposed on or integrated in the support frameand/or the substrate of the sensor arrays. Indeed, the one or moreillumination units may be disposed on or integrated in any element orcomponent of the one or more of the camera channels.

FIGS. 78C-78P are schematic views of a digital camera apparatus thatincludes one or more output devices in accordance with anotherembodiment of the present invention. More particularly, FIG. 78C is aschematic perspective view of one embodiment of a digital cameraapparatus that includes one or more output devices. In some embodiments,the one or more output devices are in the form of one or moreillumination devices, e.g., one or more illumination devices 850,however, other types of output devices may also be employed. FIGS.78C-78D are enlarged schematic front and back perspective views,respectively, of an illumination device 850 in accordance with oneembodiment of the present invention. In some embodiments, the one ormore illumination devices are in the form of one or more LED's (e.g.,one or more high power LED's).

The one or more illumination devices may be disposed in any suitablelocation or locations. In some embodiments, it may be advantageous tocollect light (for the one or more camera channels) on one side of thedigital camera assembly and to provide illumination from the same sideof the digital camera subassembly. In the illustrated embodiment, thedigital camera apparatus has first and second sides generally oppositeone another. The one or more camera channels are positioned to receivelight through a first side of the digital camera apparatus. One or moreof the illumination devices may be positioned to illuminate (e.g.,supply light) from the same side of the digital camera apparatus. Insome embodiments, such a configuration may help make it possible toprovide a digital camera apparatus that is very thin (e.g., in the zdirection). Other configurations may also be employed. In someembodiments, one or more of the illumination devices is disposed onand/or integral with one or more portions of the support, although thisis not required.

The one or more illumination devices may be connected to the processorvia one or more communication links. In some embodiments, the one ormore communication links include one or more pads on the image deviceand the one or more illumination devices and one or more electricalconnectors having one or more electrically conductive members connectingone or more of the pads on the image device to one or more of the padson the illumination devices. In some embodiments, the one or morecommunication links include one or more bump bonds that electricallyconnect one or more circuits on the image device to one or more circuitson the one or more illumination devices.

The one or more illumination devices may have any size and shape and mayor may not have the same configuration as one another (e.g., type, size,shape, resolution). In some embodiments, one or more of the one or moreillumination devices has a length and a width that are less than orequal to the length and width, respectively of the optical assembly, thesensor subassembly and/or the image device. In some embodiments, one ormore of the one or more illumination devices has a length or a widththat is greater than the length or width, respectively of the opticalassembly, the sensor subassembly and/or the image device.

FIGS. 78H-78M are schematic perspective view of digital camera apparatusin accordance with further embodiments. Such embodiments have one ormore illumination devices with configurations and/or in arrangementsthat are different than the configuration and/or arrangement of theillumination device shown in FIG. 78C. Other configurations and/orarrangements may also be employed.

As with each of the embodiments disclosed herein, this embodiment of thepresent invention may be employed alone or in combination with one ormore of the other embodiments disclosed herein, or portion thereof.

For example, FIG. 78F is a schematic perspective view of one embodimentof a digital camera apparatus that includes an output device and aspacer. FIG. 78G is a schematic perspective view of one embodiment of adigital camera apparatus that includes an output device and a spacer.The image device is shown with one or more pads connected to one or morecircuits disposed on or in the image device. FIG. 77H is a schematicperspective view of one embodiment of a digital camera apparatus thatincludes an output device, a spacer and an additional device (e.g., adisplay and/or a second device 780 adjacent to the image device). Theimage device is shown with one or more pads connected to one or morecircuits disposed on or in the image device.

Moreover, in some embodiments, the digital camera apparatus may furtherinclude a support having one or more actuators (such a support maycomprise, for example, a frame having one or more actuators) (e.g., MEMSactuators, for example, comb type MEMS actuators) to move one or more ofthe optics portions of the camera channels, one or more display devicesand/or one or more audio input devices. In some of the latterembodiments, the digital camera apparatus includes one or more audioinput devices, a support having one or more actuators (such a supportmay comprise, for example, a frame having one or more actuators) (e.g.,MEMS actuators, for example, comb type MEMS actuators) to move one ormore of the optics portions of the camera channels, one or more displaydevices and one or more illumination devices.

The digital camera apparatus may be assembled and/or mounted in anymanner, for example, but not limited to in a manner similar to thatemployed in one or more of the embodiments disclosed herein.

FIG. 78R is a schematic plan view of an underside of the support 320(e.g., a major outer surface facing toward the one or more sensorarrays) in accordance with one embodiment of the present invention. Inthis embodiment, one or more devices 850 are disposed on or in the.support 320 and receive/supply power, control signals and/or datasignals through pads 852 disposed on a surface of the support 320. Aplurality of electrical conductors (see for example, FIGS. 63-72) mayconnect one or more of the pads on the support 320 to one or morecircuits disposed elsewhere in the digital camera apparatus 300.

In some embodiments, an integrated circuit 854 may be disposed on thesupport 320 to provide, for example, but not limited to, one or morecircuits to help interface (e.g., control or communicate in any otherway) with any of devices disposed on the support 320. A plurality ofelectrically conductive traces 856 (some of which are shown) may connectthe outputs of the integrated circuit 854 to one or more of the devicesmounted on the support 320. Although shown on the surface, it should beunderstood that one, some or all of such traces may be disposed withinthe support 320 so as not to reside on the outer surface thereof.

FIGS. 79A-79C are schematic perspective views of digital cameraapparatus that include one or more input devices 830, e.g., one or moreaudio input devices (e.g., a silicon microphone) and one or more outputdevices 820, e.g., one or more display devices (e.g., a micro displaydevice), in accordance with further embodiments of the presentinvention.

FIGS. 80A-80F are schematic perspective views of digital cameraapparatus that include one or more input devices 830, e.g., one or moreaudio input devices (e.g., a silicon microphone), one or more outputdevices 820, e.g., one or more display devices (e.g., a micro displays),wherein one or more of the input devices comprise one or moreillumination devices (e.g., a high illumination LED), in accordance withfurther embodiments of the inventions.

The digital camera apparatus may be assembled and/or mounted in anymanner, for example, but not limited to in a manner similar to thatemployed in one or more of the embodiments disclosed herein.

As stated above, the digital camera apparatus may have any number ofcamera channels each of which may have any configuration. Referring toFIGS. 81A-81C, in some embodiments, the digital camera apparatusincludes a housing, for example, but not limited to a hermetic package.One or more portions of a housing may be defined by one or more of thestructures described herein, for example, one or more of the opticsportions, one or more portions of the frame, one or more portions of theimage device and/or combinations thereof.

In some embodiments, one or more portions of the housing are defined byplastic material(s), ceramic material(s) and/or any combination thereof.

FIG. 81A is a schematic perspective view a digital camera apparatus 300that includes a housing in accordance with one embodiment of the presentinvention. FIGS. 81B-81C are schematic exploded perspective views of thedigital camera apparatus 300. The housing may comprise a molded plasticpackage although this is not required. In the illustrated embodiment,the digital camera apparatus includes a first housing portion (e.g., amolded plastic base or bottom) that supports an image sensor. The imagedevice may include the one or more sensor portions and may furtherinclude one or more portions of the processor. A second housing portion(e.g., a molded plastic top or cover) defines a frame having one or moreframe portions to receive and position the one or more optics portions.One or more terminals may be provided, for example, one or moreterminals 860, and may be disposed, for example, on one or more outersurfaces of the molded plastic packaging. One or more electricallyconductive members, e.g., bond wires, may electrically connect one ormore of the terminals to one or more circuits on the image device, e.g.,one or more circuits of one or more portions of the processor. In someembodiments, the first housing portion, the second housing portion andthe one or more optics portions define a substantial portion of thehousing, for example, but not limited to a hermetic package. In someembodiments, one or more of the optics portions have a top surface thatis generally flush with one or more portions of the major outer surfaceof the second housing portion (e.g., molded plastic top).

The digital camera apparatus may be assembled in any manner. In someembodiments, the image device, terminals and electrically conductivemembers are supported superjacent the major outer surface of the firsthousing portion (e.g., a molded plastic base). The second housingportion (e.g., a molded plastic top) may be provided thereafter. Heat,pressure and/or bonding material may be employed before, during, and/orafter the assembly process. The bonding material may be any type ortypes of bonding material for example but not limited to a hermeticbonding material or materials.

The molded plastic packaging may make the digital camera subassemblymore easily removable and/or installable, for example, to facilitaterepair and/or upgrade, although this is not required. Molded plasticpackaging may also be advantageous, for example, for digital cameraapparatus employed in wearable sensors, such as, for example, a badge orbroach that does not contain a display but transmits data to a basestation. Molded plastic packaging may be employed in combination withany one or more of the embodiments disclosed herein.

Other configurations may also be employed. In some embodiments, forexample, the first housing portion and/or the second housing portion areformed of any type of hermetic material(s), for example, but not limitedto ceramic material(s). The use of ceramic packaging may be advantageousin harsh environments and/or in applications (e.g., vacuum systems)where outgassing from plastics present a problem, although this is notrequired. Ceramic packaging may be employed in combination with any oneor more of the embodiments disclosed herein.

Referring to FIG. 81D, a schematic exploded perspective view a digitalcamera apparatus that includes molded plastic packaging in accordancewith another embodiment of the present invention, in some embodiment,two digital camera apparatus are disposed in a single housing. Forexample, in some embodiments, the first housing portion (e.g., the base)defines a frame having one or more frame portions to receive andposition a second set of one or more optics portions that face in adirection opposite the one or more optics portions seated in the secondhousing portion. A second set of one or more sensor arrays may beassociated with the second set of one or more optics portions and may bedisposed, for example, on the image device or on a second image devicethat may also be disposed in the housing.

In some embodiments, one of the camera channels, e.g., camera channel350A, is dedicated to two or more separate colors or two or moreseparate bands of colors (e.g., blue or a blue band and red or a redband). In some of such embodiments, the optical portion may itself havethe capability to provide color separation, for example, similar to thatprovided by a color filter array (e.g., a Bayer pattern or variationthereof). (See for example, FIG. 82).

FIG. 82 is a schematic perspective view of a digital camera apparatushaving one or more optics portions with the capability to provide colorseparation in accordance with one embodiment of the present invention.In some of such embodiments, one or more of the optics portions, e.g.,optics portion 330C includes an array of color filters, for example, butnot limited to a Bayer patter. In some of such embodiments, one or moreof the optics portions, e.g., optics portion 330C has the capability toprovide color separation similar to that which is provided by a colorfilter array.

In some embodiments, the lens and/or filter of the camera channel maytransmit both of such colors or bands of colors and the camera channelmay include one or mechanisms elsewhere in the camera channel toseparate the two colors or two bands of colors. For example, a colorfilter array may be disposed between the lens and the sensor arrayand/or the camera channel may employ a sensor capable of separating thecolors or bands of colors. In some of the latter embodiments, the sensorarray may be provided with pixels that have multiband capability, e.g.,two or three colors. For example, each pixel may comprise two or threephotodiodes, wherein a first photodiode is adapted to detect a firstcolor or first band of colors, a second photodiode is adapted to detecta second color or band of colors and a third photodiode is adapted todetect a third color or band of colors. One way to accomplish this is toprovide the photodiodes with different structures/characteristics thatmake them selective, such that first photodiode has a higher sensitivityto the first color or first band of colors than to the second color orband of colors, and the second photodiode has a higher sensitivity tothe second color or second band of colors than to the first color orfirst band of colors. Another way is to dispose the photodiodes atdifferent depths in the pixel, which takes advantage of the differentpenetration and absorption characteristics of the different colors orbands of colors. For example, blue and blue bands of colors penetrateless (and are thus absorbed at a lesser depth) than green and greenbands of colors, which in turn penetrate less (and are thus absorbed ata lesser depth) than red and red bands of colors. In some embodiments,such a sensor array is employed even though the pixels may see only oneparticular color or band of colors, for example, to in order to adaptsuch sensor array to the particular color or band of colors. FOUR CAMERACHANNELS

In some embodiments, the digital camera apparatus includes four or morecamera channels, e.g., camera channels 350A-350D. In some of suchembodiments, the first camera channel, e.g., camera channel 350A, isdedicated to a first color or first band of colors (e.g., red or a redband of colors), the second camera channel, e.g., camera channel 350B,is dedicated to second color or second band of colors (e.g., blue or ablue band of colors), different than the first color or first band ofcolors, and the third camera channel, e.g., camera channel 350C, isdedicated to third color or third band of colors (e.g., green or a greenband of colors), different than the first and second colors or band ofcolors, and the fourth camera channel, e.g., camera channel 350C, isdedicated to fourth color or fourth band of colors (e.g., green or agreen band of colors) different than the first, second and third colorsor band of colors. In some embodiments, one or more of the camerachannels employs a pixel size that matches the color optical blur forthe respective camera channel, an integration time and/or otherelectrical characteristic of the sensor array adapted to increase oroptimize performance of the respective camera channel, and/or adesign/layout of the pixel circuitry and photodiode that is adapted toincrease or maximize sensitivity of the respective camera channel. Insome embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

The sensor arrays of the one or more camera channels may or may not havethe same field of view as one another. In some embodiments, each of thesensor arrays has the same field of view as one another. In someembodiments, one or more of the sensor arrays has a field of view thatis different field than the field of view of one or more of the othercamera channels.

In some embodiments, one of the camera channels, e.g., camera channel350A, is dedicated to two or more separate colors or two or moreseparate bands of colors (e.g., blue or a blue band and red or a redband). In some of such embodiments, the optical portion may itself havethe capability to provide color separation, for example, similar to thatprovided by a color filter array (e.g., a Bayer pattern or variationthereof) (see for example, FIG. 82). In some embodiments, the lensand/or filter of the camera channel may transmit both of such colors orbands of colors and the camera channel may include one or mechanismselsewhere in the camera channel to separate the two colors or two bandsof colors. For example, a color filter array may be disposed between thelens and the sensor array and/or the camera channel may employ a sensorcapable of separating the colors or bands of colors. In some of thelatter embodiments, the sensor array may be provided with pixels thathave multiband capability, e.g., two or three colors. For example, eachpixel may comprise two or three photodiodes, wherein a first photodiodeis adapted to detect a first color or first band of colors, a secondphotodiode is adapted to detect a second color or band of colors and athird photodiode is adapted to detect a third color or band of colors.One way to accomplish this is to provide the photodiodes with differentstructures/characteristics that make them selective, such that firstphotodiode has a higher sensitivity to the first color or first band ofcolors than to the second color or band of colors, and the secondphotodiode has a higher sensitivity to the second color or second bandof colors than to the first color or first band of colors. Another wayis to dispose the photodiodes at different depths in the pixel, whichtakes advantage of the different penetration and absorptioncharacteristics of the different colors or bands of colors. For example,blue and blue bands of colors penetrate less (and are thus absorbed at alesser depth) than green and green bands of colors, which in turnpenetrate less (and are thus absorbed at a lesser depth) than red andred bands of colors. In some embodiments, such a sensor array isemployed even though the pixels may see only one particular color orband of colors, for example, to in order to adapt such sensor array tothe particular color or band of colors.

In some of the latter embodiments, the second camera channel, e.g.,camera channel 350B, is also dedicated to two or more separate colors ortwo or more separate bands of colors. For example, the first camerachannel may be dedicated to red or a red band and green or a green band(e.g., G1). The second camera channel may be dedicated to blue or a blueband and green or a green band (e.g., G2). In some other of the latterembodiments, the second camera channel, e.g., camera channel 350B, isdedicated to a single color or single band of colors (e.g., green or agreen band) different from the colors or bands of colors to which thefirst camera channel is dedicated, and the third camera channel, e.g.,camera channel 350C, is dedicated to a single color or single band ofcolors different from the colors or bands of colors to which the firstand second camera channels are dedicated.

The camera channels may or may not have the same configuration (e.g.,size, shape, resolution, or degree or range of sensitivity) as oneanother. In some embodiments, for example, each of the camera channelshas the same size, shape, resolution and/or a degree or range ofsensitivity as the other camera channels. In some other embodiments, oneor more of the camera channels has a size, shape, resolution and/or adegree or range of sensitivity that is different than one or more of theother camera channels. In that regard, in some embodiments, each of thecamera channels, e.g., camera channels 350A-350D, has the sameresolution as one another. In some other embodiments, one or more of thecamera channels has a resolution that is less than the resolution of oneor more of the other camera channels. For example, one or more of thecamera channels, e.g., camera channel 350A, may have a sensor array withfewer pixels than the sensor array of one or more of the other camerachannels, e.g., camera channel 350B, for a comparable portion of thefield of view. In one embodiment, for example, the number of pixels inone of the camera channels is forty four percent greater than the numberof pixels in another camera channel, for a comparable portion of thefield of view. In another embodiment, for example, the number of pixelsin one of the camera channels is thirty six percent greater than thenumber of pixels in the other camera channels, for a comparable portionof the field of view.

In some embodiments, one or more of the camera channels may have asensor array with a size that is different than the size of the sensorarray of one or more of the other camera channels. In some of suchembodiments, the optics portion of such one or more camera channels mayhave a f/# and/or a focal length that is different from the f/# and/or afocal length of the one or more of the other camera channels.

In some embodiments, one or more of the camera channels are dedicated toa wavelength or band of wavelengths and the sensor array and/or opticsportion of such one or more camera channels are optimized for therespective wavelength or band of wavelengths to which the respectivecamera channel is dedicated. In some embodiments, the design, operation,array size and/or pixel size of each sensor array is optimized for therespective wavelength or bands of wavelengths to which the camerachannels are dedicated. In some embodiments the design of each opticalportion is optimized for the respective wavelength or bands ofwavelengths to which the respective camera channel is dedicated.

It should be understood, however, that any other configurations may alsobe employed.

The four or more camera channels may be arranged in any manner. In someembodiments, the four or more camera channels are arranged in a 2×2matrix to help provide compactness and symmetry in optical collection.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as or similar to one or more of the layouts described herein(see for example, FIGS. 83A-83C). In some embodiments, the processor mayhave one or more portions that are not integrated on the same integratedcircuit as the sensor arrays and/or may not have any portions disposedon the same integrated circuit as the sensor arrays (see for example,FIGS. 83D-83E).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the camera channels, e.g., camera channels350A-350D, are connected via one or more communication links to one ormore displays. In some such embodiments, each of the camera channels isconnected to its own display. The displays may or may not have the samecharacteristics as one another. In some other embodiments, the fourcamera channels, e.g., camera channels 350A-350D, are each connected tothe same display.

Four Camera Channels in Y Configuration

FIGS. 84A-84E are schematic representations of digital camera apparatus300 in accordance with further embodiments of the present invention. Ineach of these embodiments, the digital camera apparatus includes four ormore camera channels, e.g., camera channels 350A-350D, wherein four ofthe camera channels, e.g., camera channels 350A-350D, are arranged in a“Y” configuration.

In some embodiments, one of the camera channels, e.g., camera channel350C, is a broadband camera channel, an infrared (IR) camera channel oran ultraviolet (UV) camera channel.

In some embodiments, a first one of the camera channels, e.g., camerachannel 350A, is dedicated to a first color or first band of colors(e.g., red or a red band of colors), a second one the camera channels,e.g., camera channel 350B, is dedicated to second color or second bandof colors (e.g., blue or a blue band of colors), different than thefirst color or first band of colors, and a third one of the camerachannels, e.g., camera channel 350D, is dedicated to a third color orthird band of colors (e.g., green or a green band of colors), differentthan the first and second colors or band of colors. In some of suchembodiments, the other camera channel, e.g., camera channels 350D, is abroadband camera channel, e.g., using a color filter array with a Bayerpattern, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, one of the camera channels, e.g., camera channel350C, is dedicated to two or more colors or two or more bands of colors.In some of such embodiments, the optical portion may itself have thecapability to provide color separation, for example, similar to thatprovided by a color filter array (e.g., a Bayer pattern or variationthereof) (see for example, FIG. 84B). In some embodiments, the lensand/or filter of the camera channel may transmit both of such colors orbands of colors and the camera channel may include one or mechanismselsewhere in the camera channel to separate the two colors or two bandsof colors. For example, a color filter array may be disposed between thelens and the sensor array and/or the camera channel may employ a sensorcapable of separating the colors or bands of colors. In some of thelatter embodiments, the sensor array may be provided with pixels thathave multiband capability, e.g., two or three colors. For example, eachpixel may comprise two or three photodiodes, wherein a first photodiodeis adapted to detect a first color or first band of colors, a secondphotodiode is adapted to detect a second color or band of colors and athird photodiode is adapted to detect a third color or band of colors.One way to accomplish this is to provide the photodiodes with differentstructures/characteristics that make them selective, such that firstphotodiode has a higher sensitivity to the first color or first band ofcolors than to the second color or band of colors, and the secondphotodiode has a higher sensitivity to the second color or second bandof colors than to the first color or first band of colors. Another wayis to dispose the photodiodes at different depths in the pixel, whichtakes advantage of the different penetration and absorptioncharacteristics of the different colors or bands of colors. For example,blue and blue bands of colors penetrate less (and are thus absorbed at alesser depth) than green and green bands of colors, which in turnpenetrate less (and are thus absorbed at a lesser depth) than red andred bands of colors. In some embodiments, such a sensor array isemployed even though the pixels may see only one particular color orband of colors, for example, to in order to adapt such sensor array tothe particular color or band of colors.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as or similar to one or more of the layouts described herein(see for example, FIGS. 84C-84E). In some embodiments, one, some or allportions of the processor are not disposed on the same integratedcircuit as the sensor arrays. (see for example, FIG. 84A).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

Four Channels Two Smaller Than Other Two

FIGS. 85A-85E are schematic representations of digital camera apparatus300 in accordance with further embodiments of the present invention. Ineach of these embodiments, the digital camera apparatus includes four ormore camera channels, e.g., camera channels 350A-350D. Two of the camerachannels, e.g., camera channels 350A, 350C, are each smaller in sizethan two of the other camera channel, e.g., camera channel 350B, 350D.

In some embodiments, the smaller camera channels, e.g., camera channels350A, 350C, each have a resolution that is less than the resolution ofthe larger camera channels, e.g., camera channel 350B, 350D, although insuch embodiments, the smaller camera channels may or may not have thesame resolution as one another and the larger camera channels may or maynot have the same resolution as one another. For example, each of thesmaller camera channels, may have a sensor array with fewer pixels thanis provided in the sensor array of each of the larger camera channelsfor a comparable portion of the field of view. In one embodiment, forexample, the number of pixels in one or more of the larger camerachannels is forty four percent greater than the number of pixels in oneor more of the smaller camera channels, for a comparable portion of thefield of view. In another embodiment, for example, the number of pixelsin one or more of the larger camera channels is thirty six percentgreater than the number of pixels in one or more of the smaller camerachannels, for a comparable portion of the field of view. It should beunderstood, however, that any other sizes and/or architectures may alsobe employed.

In some other embodiments, one or more of the smaller camera channelshas a resolution that is equal to the resolution of one or more of thelarger camera channels. For example, one or more of the smaller camerachannels, e.g., camera channels 350A, 350C, may have a sensor array withthe same number of pixels as is provided in the sensor array of thelarger camera channels, e.g., camera channel 350B, 350D, for acomparable portion of the field of view. In one embodiment, for example,the pixels in the larger camera channels are forty four percent largerin size (e.g., twenty percent larger in the x direction and twentypercent larger in the y direction) than the pixels in the smaller camerachannels. In another embodiment, for example, the pixels in the largercamera channels are thirty six percent larger in size (e.g., seventeenpercent larger in the x direction and seventeen percent larger in the ydirection) than the pixels in the smaller camera channels. It should beunderstood, however, that any other sizes and/or architectures may alsobe employed.

In some embodiments, one or more of the camera channels may have asensor array with a size that is different than the size of the sensorarray of one or more of the other camera channels. In some of suchembodiments, the optics portion of such one or more camera channels mayhave a f/# and/or a focal length that is different from the f/# and/or afocal length of the one or more of the other camera channels.

In some embodiments, one of the smaller camera channels, e.g., camerachannel 350A, is dedicated to a first color or first band of colors(e.g., red or a red band of colors), one of the larger camera channel,e.g., camera channel 350B, is dedicated to second color or second bandof colors (e.g., blue or a blue band of colors), different than thefirst color or first band of colors, and the other larger camerachannel, e.g., camera channel 350D, is dedicated to third color or thirdband of colors (e.g., green or a green band of colors), different thanthe first and second colors or band of colors. In some of suchembodiments, the smaller camera channel, e.g., camera channels 350A, hasa resolution that is equal to the resolution of the two larger camerachannels, e.g., camera channels 350B, 350D.

In some embodiments, one of the smaller camera channels, e.g., camerachannel 350C, is a broadband camera channel, an infrared (IR) camerachannel or an ultraviolet (UV) camera channel.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as or similar to one or more of the layouts described herein(see for example, FIGS. 85C-85E). In some embodiments, the processor mayhave one or more portions that are not disposed on the same integratedcircuit as the sensor arrays and/or may not have any portions disposedon the same integrated circuit as the sensor arrays (see for example,FIG. 85B).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the four or more camera channels, e.g., camerachannels 350A-350D, are connected via one or more communication links toone or more displays. In some such embodiments, each of the camerachannels is connected to its own display. The displays may or may nothave the same characteristics as one another. In some other embodiments,two or more camera channels, e.g., camera channels 350A-350B, 350D, areconnected to a first display and one or more of the other camerachannels, e.g., camera channel 350C, is connected to a second display.The first and second displays may or may not have the samecharacteristics. In some such embodiments, the first display has aresolution equal to the resolution of one or more of the camera channelsconnected thereto. The second display may have a resolution equal to theresolution of one or more camera channels connected thereto. Forexample, in some embodiments, one or more of the camera channels have aresolution that is less than the resolution of one or more other camerachannels. In such embodiments, the one or more display connected to theone or more lower resolution camera channels may have aresolution/resolutions that is/are lower than the resolution/resolutionsof the one or more displays connected to the one or more other camerachannels. In some embodiments, the first display has a resolution equalto the resolution of one or more of the camera channels connectedthereto. The second display may have a resolution equal to theresolution of one or more camera channels connected thereto, however,other resolutions may be employed.

Four Channels, Three Smaller Than Other

FIGS. 86A-86E are schematic representations of digital camera apparatus300 in accordance with further embodiments of the present invention. Ineach of these embodiments, the digital camera apparatus includes four ormore camera channels, e.g., camera channels 350A-350D. Three of thecamera channels, e.g., camera channels 350A-350C, are each smaller insize than the other camera channel, e.g., camera channel 350D.

In some embodiments, the smaller camera channels, e.g., camera channels350A-350C, each have a resolution that is less than the resolution ofthe larger camera channel, e.g., camera channel 350D, although in suchembodiments, the smaller camera channels may or may not have the sameresolution as one another. For example, each of the smaller camerachannels may have a sensor array with fewer pixels than is provided inthe sensor array of the larger camera channel for a comparable portionof the field of view. In one embodiment, for example, the number ofpixels in the larger camera channels is forty four percent greater thanthe number of pixels in one or more of the smaller camera channels, fora comparable portion of the field of view. In another embodiment, forexample, the number of pixels in the larger camera channel is thirty sixpercent greater than the number of pixels in one or more of the smallercamera channels, for a comparable portion of the field of view. Itshould be understood, however, that any other sizes and/or architecturesmay also be employed.

In some other embodiments, one or more of the smaller camera channelshas a resolution that is equal to the resolution of the larger camerachannel. For example, one or more of the smaller camera channels, e.g.,camera channels 350A-350C, may have a sensor array with the same numberof pixels as is provided in the sensor array of the larger camerachannel, e.g., camera channel 350D, for a comparable portion of thefield of view. In one embodiment, for example, the pixels in the largercamera channel are forty four percent larger in size (e.g., twentypercent larger in the x direction and twenty percent larger in the ydirection) than the pixels in the smaller camera channels. In anotherembodiment, for example, the pixels in the larger camera channel arethirty six percent larger in size (e.g., seventeen percent larger in thex direction and seventeen percent larger in the y direction) than thepixels in the smaller camera channels. It should be understood, however,that any other sizes and/or architectures may also be employed.

In some embodiments, one of the camera channels, e.g., camera channel350D, is a broadband camera channel, an infrared (IR) camera channel oran ultraviolet (UV) camera channel.

In some embodiments, a first one of the camera channels, e.g., camerachannel 350A, is dedicated to a first color or first band of colors(e.g., red or a red band of colors), a second one the camera channels,e.g., camera channel 350B, is dedicated to second color or second bandof colors (e.g., blue or a blue band of colors), different than thefirst color or first band of colors, and a third one of the camerachannels, e.g., camera channel 350C, is dedicated to a third color orthird band of colors (e.g., green or a green band of colors), differentthan the first and second colors or band of colors. In some of suchembodiments, the other camera channel, e.g., camera channels 350D, is abroadband camera channel, e.g., using a color filter array with a Bayerpattern.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, FIGS. 86C-86E). In some embodiments, the processor may have oneor more portions that are not disposed on the same integrated circuit asthe sensor arrays and/or may not have any portions disposed on the sameintegrated circuit as the sensor arrays (see for example, FIG. 86B).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the four or more camera channels, e.g., camerachannels 350A-350D, are connected via one or more communication links toone or more displays. In some such embodiments, each of the camerachannels is connected to its own display. The displays may or may nothave the same characteristics as one another. In some other embodiments,three or more camera channels, e.g., camera channels 350A-350C, areconnected to a first display and the other camera channel, e.g., camerachannel 350D, is connected to a second display. The first and seconddisplays may or may not have the same characteristics as one another.For example, in some embodiments, one or more of the camera channelshave a resolution that is less than the resolution of one or more othercamera channels. In such embodiments, the one or more display connectedto the one or more lower resolution camera channels may have aresolution/resolutions that is/are lower than the resolution/resolutionsof the one or more displays connected to the one or more other camerachannels. In some embodiments, the first display has a resolution equalto the resolution of one or more of the camera channels connectedthereto. The second display may have a resolution equal to theresolution of one or more camera channels connected thereto, however,other resolutions may be employed.

Four Elliptical Channels

FIGS. 87A-87B are schematic representation of digital camera apparatus300 in accordance with further embodiments of the present invention. Ineach of these embodiments, the digital camera apparatus includes one ormore camera channels, e.g., camera channels 350A-350D, one or more ofwhich having optical portion, e.g., optical portions 330A-330D,respectively, with an elliptical or other non circular shape.

In some embodiments, a first one of the camera channels, e.g., camerachannel 350A, is dedicated to a first color or first band of colors(e.g., red or a red band of colors), a second one the camera channels,e.g., camera channel 350B, is dedicated to second color or second bandof colors (e.g., blue or a blue band of colors), different than thefirst color or first band of colors, and a third one of the camerachannels, e.g., camera channel 350D, is dedicated to a third color orthird band of colors (e.g., green or a green band of colors), differentthan the first and second colors or band of colors. In some of suchembodiments, the other camera channel, e.g., camera channels 350D, is abroadband camera channel, e.g., using a color filter array with a Bayerpattern, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some other embodiments, a first one of the camera channels, e.g.,camera channel 350A, is dedicated to a first color or first band ofcolors (e.g., red or a red band of colors), a second one the camerachannels, e.g., camera channel 350B, is dedicated to second color orsecond band of colors (e.g., green or a green band of colors), differentthan the first color or first band of colors, a third one of the camerachannels, e.g., camera channel 350C, is dedicated to a third color orthird band of colors (e.g., blue or a blue band of colors), differentthan the first and second colors or band of color, and a fourth one ofthe camera channels, e.g., camera channel 350D, is dedicated to a coloror band of colors (e.g., green or a green band of colors), differentthan the first and third colors or bands of colors.

In some other embodiments, a first one of the camera channels, e.g.,camera channel 350A, is dedicated to a red or a red band of colors, asecond one the camera channels, e.g., camera channel 350B, is dedicatedto blue or a blue band of colors, a third one of the camera channels,e.g., camera channel 350C, is dedicated to green1 or a green1 band ofcolors, and a fourth one of the camera channels, e.g., camera channel350D, is dedicated to a green2 or green2 band of colors.

In some embodiments, one of the camera channels, e.g., camera channel350C, is a broadband camera channel, an infrared (IR) camera channel oran ultraviolet (UV) camera channel.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein. In someembodiments, the processor may have one or more portions that are notdisposed on the same integrated circuit as the sensor arrays and/or maynot have any portions disposed on the same integrated circuit as thesensor arrays (see for example, FIGS. 87A-87B).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

Three Camera Channels

FIGS. 88A-88E and 89A-89E are schematic representation of digital cameraapparatus 300 in accordance with further embodiments of the presentinvention. In each of these embodiments, the digital camera apparatusincludes three or more camera channels, e.g., camera channels 350A-350C.

In some embodiments, the first camera channel, e.g., camera channel350A, is dedicated to a first color or first band of colors (e.g., redor a red band of colors), the second camera channel, e.g., camerachannel 350B, is dedicated to second color or second band of colors(e.g., blue or a blue band of colors), different than the first color orfirst band of colors, and the third camera channel, e.g., camera channel350C, is dedicated to third color or third band of colors (e.g., greenor a green band of colors), different than the first and second colorsor band of colors. In some embodiments, one or more of the camerachannels employs a pixel size that matches a color optical blur for therespective camera channel, an integration time and/or other electricalcharacteristic of the sensor array adapted to increase or optimizeperformance of the respective camera channel, and/or a design/layout ofthe pixel circuitry and photodiode that is adapted to increase ormaximize sensitivity of the respective camera channel.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, one of the camera channel, e.g., camera channel350A, is dedicated to two or more separate colors or two or moreseparate bands of colors (e.g., blue or a blue band and red or a redband). In some of such embodiments, the optical portion may itself havethe capability to provide color separation, for example, similar to thatprovided by a color filter array (e.g., a Bayer pattern or variationthereof). In some embodiments, the lens and/or filter of the camerachannel may transmit both of such colors or bands of colors and thecamera channel may include one or mechanisms elsewhere in the camerachannel to separate the two colors or two bands of colors. For example,a color filter array may be disposed between the lens and the sensorarray and/or the camera channel may employ a sensor capable ofseparating the colors or bands of colors. In some of the latterembodiments, the sensor array may be provided with pixels that havemultiband capability, e.g., two or three colors. For example, each pixelmay comprise two or three photodiodes, wherein a first photodiode isadapted to detect a first color or first band of colors, a secondphotodiode is adapted to detect a second color or band of colors and athird photodiode is adapted to detect a third color or band of colors.One way to accomplish this is to provide the photodiodes with differentstructures/characteristics that make them selective, such that firstphotodiode has a higher sensitivity to the first color or first band ofcolors than to the second color or band of colors, and the secondphotodiode has a higher sensitivity to the second color or second bandof colors than to the first color or first band of colors. Another wayis to dispose the photodiodes at different depths in the pixel, whichtakes advantage of the different penetration and absorptioncharacteristics of the different colors or bands of colors. For example,blue and blue bands of colors penetrate less (and are thus absorbed at alesser depth) than green and green bands of colors, which in turnpenetrate less (and are thus absorbed at a lesser depth) than red andred bands of colors. In some embodiments, such a sensor array isemployed even though the pixels may see only one particular color orband of colors, for example, to in order to adapt such sensor array tothe particular color or band of colors.

In some of the latter embodiments, the second camera channel, e.g.,camera channel 350B, may also be dedicated to two or more separatecolors or two or more separate bands of colors. For example, the firstcamera channel may be dedicated to red or a red band and green or agreen band (e.g., G1). The second camera channel may be dedicated toblue or a blue band and green or a green band (e.g., G2). In some otherof the latter embodiments, the second camera channel, e.g., camerachannel 350B, may be dedicated to a single color or single band ofcolors (e.g., green or a green band) different from the colors or bandsof colors to which the first camera channel is dedicated, and the thirdcamera channel, e.g., camera channel 350C, is dedicated to a singlecolor or single band of colors different from the colors or bands ofcolors to which the first and second camera channels are dedicated.

The three or more camera channels may or may not have the sameconfiguration (e.g., size, shape, resolution, or degree or range ofsensitivity) as one another. In some embodiments, each of the camerachannels has the same size, shape, resolution and/or a degree or rangeof sensitivity as the other camera channels. In some other embodiments,for example, one or more of the camera channels has a size, shape,resolution and/or a degree or range of sensitivity that is differentthan one or more of the other camera channels. For example, one or moreof the camera channels may have a sensor array with fewer pixels than isprovided in the sensor array of one or more of the other camerachannels, for a comparable portion of a field of view.

In some embodiments, one or more of the camera channels may have asensor array with a size that is different than the size of the sensorarray of one or more of the other camera channels. In some of suchembodiments, the optics portion of such one or more camera channels mayhave a f/# and/or a focal length that is different from the f/# and/or afocal length of the one or more of the other camera channels.

In some embodiments, one or more of the camera channels are dedicated toa wavelength or band of wavelengths and the sensor array and/or opticsportion of such one or more camera channels are optimized for therespective wavelength or band of wavelengths to which the respectivecamera channel is dedicated. In some embodiments, the design, operation,array size and/or pixel size of each sensor array is optimized for therespective wavelength or bands of wavelengths to which the camerachannels are dedicated. In some embodiments the design of each opticalportion is optimized for the respective wavelength or bands ofwavelengths to which the respective camera channel is dedicated.

It should be understood, however, that any other configurations may beemployed.

The three or more camera channels may be arranged in any manner. In someembodiments, the three or more camera channels are arranged in atriangle, as shown to help provide compactness and symmetry in opticalcollection.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, FIGS. 98A-98B. In some embodiments, the processor may have oneor more portions that are not disposed on the same integrated circuit asthe sensor arrays and/or may not have any portions disposed on the sameintegrated circuit as the sensor arrays.

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the camera channels, e.g., camera channels350A-350C, are connected via one or more communication links to one ormore displays. In some such embodiments, each of the camera channels isconnected to its own display. The displays may or may not have the samecharacteristics as one another. In some other embodiments, the threecamera channels, e.g., camera channels 350A-350C, are each connected tothe same display.

FIGS. 90A, 91A-91B, 92A-92B, 93A-93, 94A-94B, 95A-95B and 96A-96B are aschematic plan views and a schematic cross sectional view, respectively,of some embodiments of the image device 520 that may be employed inassociation with a digital camera apparatus having three or more camerachannels. In this embodiment, the image device has first and secondmajor surfaces and an outer perimeter defined by edges. The image devicedefines the one or more regions for the active areas of the one or moresensor arrays. The image device further defines one or more regions forthe buffer and/or logic associated with the one or more sensor arrays.

The image device, sensor arrays and image areas may each have anysize(s) and shape(s). In some embodiments, the image areas are generallyabout the same size as the respective sensor arrays, and therefore, theimage areas may differ from one another in size and shape depending uponthe dimensions of the underlying sensor arrays. Of course, there is norequirement that an image area cover all, or only, the underlying array.In alternative embodiments an image area could cover only a portion ofan array, and could extend beyond the array.

The image device 520 has a generally rectangular shape and dimensionsequal to about 10 mm on a first side and about 8.85 mm on a second side.Each of the image areas has a generally circular shape and a width ordiameter equal to about 5 mm. Each of the active areas has a generallyrectangular shape having a first dimension equal to about 4.14 mm and asecond dimension equal to about 3.27 mm. The active area, may define,for example, a matrix of 1200×900 pixels (i.e., 1200 columns, 900 rows).

In some embodiments, the image device 520 has a generally square shapeand dimensions equal to about 10 mm on a side, with each quadrant being5 mm on a side. Each of the image areas has a generally circular shapeand a width or diameter equal to about 5 millimeters (mm). Each of theactive areas has a generally rectangular shape having a first dimensionequal to about 4mm and a second dimension equal to about 3mm. The activearea, may define for example, a matrix of 1200×900 pixels (i.e., 1200columns, 900 rows).

Referring to FIGS. 97A-97D, the optics portions of the three or morecamera channels are supported by one or more supports, e.g., support320, which position each of the optics portions in registration with arespective sensor array, at least in part. In this embodiment, forexample, optics portion 330A is positioned in registration with sensorarray 310A. Optics portion 330B is positioned in registration withsensor array 310B. Optics portion 330C is positioned in registrationwith sensor array 310C. Optics portion 330B is positioned inregistration with sensor array 310B. In some embodiments, the supportalso helps to limit, minimize and/or eliminate light “cross talk”between the camera channels.

In this embodiment, the support 320 includes a support that defines oneor more support portions, e.g., four support portions 600A-600C, each ofwhich supports and/or helps position a respective one of the one or moreoptics portions. In this embodiment, for example, support portion 600Asupports and positions optics portion 330A in registration with sensorarray 310A. Support portion 600B supports and positions optics portion330B in registration with sensor array 310B. Support portion 600Csupports and positions optics portion 330C in registration with sensorarray 310C. Support portion 600D supports and positions optics portion330D in registration with sensor array 310D. In this embodiment, thesupport also helps to limit, minimize and/or eliminate light “crosstalk” between the camera channels.

Each of the support portions 600A-600C defines an aperture 616 and aseat 618. The aperture 616 defines a passage for the transmission oflight for the respective camera channel. The seat 618 is adapted toreceive a respective one of the optics portions (or portion thereof) andto support and/or position the respective optics portion, at least inpart. In this regard, the seat 618 may include one or more surfaces(e.g., surfaces 620, 622) adapted to abut one or more surfaces of theoptics portion to support and/or position the optics portion, at leastin part, relative to the support portion and/or one or more of thesensor arrays 310A-310C. In this embodiment, surface 620 is disposedabout the perimeter of the optics portion to support and help positionthe optics portion in the x direction and the y direction). Surface 622(sometimes referred to herein as “stop” surface) positions or helpsposition the optics portion in the z direction.

The position and/or orientation of the stop surface 622 may be adaptedto position the optics portion at a specific distance (or range ofdistance) and/or orientation with respect to the respective sensorarray. In this regard, the seat 618 controls the depth at which the lensis positioned (e.g., seated) within the support. The depth may bedifferent for each lens and is based, at least in part, on the focallength of the lens. For example, if a camera channel is dedicated to aspecific color (or band of colors), the lens or lenses for that camerachannel may have a focal length specifically adapted to the color (orband of colors) to which the camera channel is dedicated. If each camerachannels is dedicated to a different color (or band of colors) than theother camera channels, then each of the lenses may have a differentfocal length, for example, to tailor the lens to the respective sensorarray, and each of the seats have a different depth.

Each optics portion may be secured in the respective seat 618 in anysuitable manner, for example, but not limited to, mechanically (e.g.,press fit, physical stops), chemically (e.g., adhesive), electronically(e.g., electronic bonding) and/or any combination thereof. In someembodiments, the seat 618 has dimensions adapted to provide a press fitfor the respective optics portion.

The aperture (or portions thereof) may have any configuration (e.g.,shape and/or size) including for example, cylindrical, conical,rectangular, irregular and/or any combination thereof. The configurationmay be based, for example, on the desired configuration of the opticalpath, the configuration of the respective optical portion, theconfiguration of the respective sensor array and/or any combinationthereof.

The supports 320 may comprise any type of material(s) and may have anyconfiguration and/or construction. In some embodiments, for example, thesupport 320 comprises silicon, semiconductor, glass, ceramic, plastic,or metallic materials and/or a combination thereof. If the support 320has more than one portion, such portions may be fabricated separate fromone another, integral with one another and/or any combination thereof.If the support defines more than one support portion, each of suchsupport portions, e.g., support portions 600A-600D, may be coupled toone, some or all of the other support portions, as shown, or completelyisolated from the other support portions. If the support 320 is a singleintegral component, each of the one or more support portions define oneor more portions of such integral component. Moreover, the positionermay be a solid device that may offer a wide range of options formanufacturing and material, however other forms of devices may also beemployed. In some embodiments, for example, the support 320 comprises aplate (e.g., a thin plate) that defines the support and one or moresupport portions, with the apertures and seats being formed by machining(e.g., boring) or any other suitable manner. In some other embodiments,the support 320 is fabricated as a casting with the apertures definedtherein (e.g., using a mold with projections that define the aperturesand seats of the one or more support portions).

In some embodiments, the lens and support are manufactured as a singlemolded component. In some embodiments the lens may be manufactured withtabs that may be used to form the support.

In some embodiments, the support 320 is coupled and/or affixed directlyor indirectly, to the image device. For example, the support 320 may bedirectly coupled and affixed to the image device (e.g., using adhesive)or indirectly coupled and/or or affixed to the image device via anintermediate support member (not shown).

The x and y dimensions of the support 320 may be, for example,approximately the same (in one or more dimensions) as the image device,approximately the same (in one or more dimensions) as the arrangement ofthe optics portions 330A-330D and/or approximately the same (in one ormore dimensions) as the arrangement of the sensor arrays 310A-310D. Oneadvantage of such dimensioning is that it helps keep the x and ydimensions of the digital camera apparatus as small as possible.

In some embodiments, it may be advantageous to provide the seat 618 witha height A that is the same as the height of a portion of the opticsthat will abut the stop surface 620. It may be advantageous for the stopsurface 622 to be disposed at a height B (e.g., the distance between thestop surface 622 and the base of the support portion) that is at leastas high as needed to allow the seat 618 to provide a firm stop for anoptics portion (e.g., the lens) to be seated thereon. The width ordiameter C of the portion of the aperture 616 disposed above the heightof the stop surface 622 may be based, for example, on the width ordiameter of the optics portion (e.g., the lens) to be seated therein andthe method used to affix and/or retain that optics portion in the seat618. The width of the stop surface 622 is preferably large enough tohelp provide a firm stop for the optics portion (e.g., the lens) yetsmall enough to minimize unnecessary blockage of the light transmittedby the optics portion. It may be desirable to make the width or diameterD of the portion of the aperture 616 disposed below the height of thestop surface 622 large enough to help minimize unnecessary blockage ofthe light transmitted by the optics portion. It may be desirable toprovide the support with a height E equal to the minimum dimensionneeded to result in a support sturdy enough to support the one or moreoptics portions to be seated therein, in view of the considerationsabove, and may be advantageous to space the one or more apertures616A-616D of the one or more support portions 600A-600D by a distance Fthat is as small as possible yet large enough that the support will besturdy enough to support the optics portions to be seated therein. Thesupport may have a length J and a width K.

In some embodiments, it is desirable to provide the seat 618 with aheight A equal to 2.2 mm, to provide the stop surface 622 at a height Bin the range of from 0.25 mm to 3 mm, to make the width or diameter C ofthe portion of the aperture above the height B of the stop surface 622equal to approximately 3 mm, to make the width or diameter D of thelower portion of the aperture approximately 2.8 mm, to provide thesupport portion with a height E in the range from 2.45 mm to 5.2 mm andto space the apertures apart by a distance F of at least 1 mm. In someof such embodiments, it may be desirable to provide the support with alength J equal to 10 mm and a width K equal to 10 mm or to provide thesupport with a length J equal to 10 mm and a width K equal to 8.85 mm.

In some embodiments, one or more of the optics portions comprises acylindrical type of lens, e.g., a NT45-090 lens manufactured by EdmundsOptics. Such lens has a cylindrical portion with a diameter G up to 3millimeters (mm) and a height H of 2.19 mm. In such embodiments, it maybe desirable to employ a support having the dimensions and ranges setforth in the paragraph above.

In some embodiments, the support has a length J equal to 10 mm and awidth K equal to 10 mm. In some other embodiments, it may be desirableto provide the support with a length J equal to 10 mm and a width Kequal to 8.85 mm.

Three Channels Two Smaller Than Other

FIGS. 99A-99D are schematic representations of digital camera apparatus300 in accordance with further embodiments of the present invention. Ineach of these embodiments, the digital camera apparatus include three ormore camera channels, e.g., camera channels 350A-350C. Two of the camerachannels, e.g., camera channels 350A-350B, are each smaller in size thana third camera channel, e.g., camera channel 350C. The smaller camerachannels may or may not have the same size as one another.

In some embodiments, the smaller camera channels, e.g., camera channels350A-350B, each have a resolution that is less than the resolution ofthe larger camera channel, e.g., camera channel 350C, although in suchembodiments, the smaller camera channels may or may not have the sameresolution as one another. For example, each of the smaller camerachannels, e.g., camera channel 350A-350B, may have a sensor array withfewer pixels than is provided in the sensor array of the larger camerachannel, e.g., camera channel 350B, for a comparable portion of thefield of view. In one embodiment, for example, the number of pixels inthe larger camera channel is forty four percent greater than the numberof pixels in one or more of the smaller camera channels, for acomparable portion of the field of view. In another embodiment, forexample, the number of pixels in the larger camera channel is thirty sixpercent greater than the number of pixels in one or more of the smallercamera channels, for a comparable portion of the field of view. Itshould be understood, however, that any other sizes and/or architecturesmay also be employed.

In some other embodiments, one or more of the smaller camera channelshas a resolution that is equal to the resolution of the larger camerachannel. For example, one or more of the smaller camera channels, e.g.,camera channels 350A-350B, may have a sensor array with the same numberof pixels as is provided in the sensor array of the larger camerachannel, e.g., camera channel 350C, for a comparable portion of thefield of view. In one embodiment, for example, the pixels in the largercamera channel are forty four percent larger in size (e.g., twentypercent larger in the x direction and twenty percent larger in the ydirection) than the pixels in the smaller camera channels. In anotherembodiment, for example, the pixels in the larger camera channel arethirty six percent larger in size (e.g., seventeen percent larger in thex direction and seventeen percent larger in the y direction) than thepixels in the smaller camera channels. It should be understood, however,that any other sizes and/or architectures may also be employed.

In some embodiments, the first camera channel, e.g., camera channel350A, is dedicated to a first color or first band of colors (e.g., redor a red band of colors), the second camera channel, e.g., camerachannel 350B, is dedicated to second color or second band of colors(e.g., blue or a blue band of colors), different than the first color orfirst band of colors, and the third camera channel, e.g., camera channel350C, is dedicated to third color or third band of colors (e.g., greenor a green band of colors), different than the first and second colorsor band of colors. In some of such embodiments, the two smaller camerachannels, e.g., camera channels 350A-350B, each have a lower resolutionthan the third camera channel, e.g., camera channel 350C.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, 99B-99D). In some embodiments, the processor may have one ormore portions that are not disposed on the same integrated circuit asthe sensor arrays and/or may not have any portions disposed on the sameintegrated circuit as the sensor arrays (see for example, FIG. 89A).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the camera channels, e.g., camera channels350A-350C, are connected via one or more communication links to one ormore displays. In some such embodiments, each of the camera channels isconnected to its own display. The displays may or may not have the samecharacteristics as one another. In some other embodiments, the smallercamera channels, e.g., camera channels 350A-350B, are connected to afirst display and the larger camera channel, e.g., camera channel 350C,is connected to a second display. The first and second displays may ormay not have the same characteristics. In some embodiments, the firstdisplay has a resolution equal to the resolution of one or more of thecamera channels connected thereto. The second display may have aresolution equal to the resolution of one or more camera channelsconnected thereto, however, other resolutions may be employed.

Three Channels of all Different Sizes

FIGS. 100A-100D are schematic representations of digital cameraapparatus 300 in accordance with further embodiments of the presentinvention. In each of these embodiments, the digital camera apparatusinclude three or more camera channels, e.g., camera channels 350A-350C.A first one of the camera channels, e.g., camera channel 350A, issmaller in size than a second one of the camera channels, e.g., camerachannel 350B, which is in turn smaller in size than a third one of thecamera channels, e.g., camera channel 350C.

Referring to FIGS. 101A-101G, in some embodiments, the smallest camerachannels, e.g., camera channels 350A, has a resolution that is smallerthan the resolution of the second camera channel, e.g., camera channel350B, which has a resolution that is smaller than the resolution of thelargest camera channel, e.g., camera channel 350C. For example, thesmallest camera channels, e.g., camera channel 350A, may have a sensorarray with fewer pixels than is provided in the sensor array of thesecond camera channel, e.g., camera channel 350B, for a comparableportion of the field of view, and the second camera channel, e.g.,camera channel 350B, may have a sensor array with fewer pixels than isprovided in the sensor array of the largest camera channel, e.g., camerachannel 350C, for a comparable portion of the field of view. In oneembodiment, for example, the number of pixels in the second camerachannel, e.g., camera channel 350B, is forty four percent greater thanthe number of pixels in the smallest camera channel, e.g., camerachannel 350A, for a comparable portion of the field of view, and thenumber of pixels in the largest camera channel, e.g., camera channel350C, is thirty six percent greater than the number of pixels in thesecond camera channel, e.g., camera channel 350B, for a comparableportion of the field of view. It should be understood, however, that anyother sizes and/or architectures may also be employed.

Referring to FIGS. 102A-102G, in some other embodiments, one or more ofthe smaller camera channels, e.g., camera channels 350A-350B, has aresolution that is equal to the resolution of the larger camera channel,e.g., camera channel 350C. For example, one or more of the smallercamera channels, e.g., camera channels 350A-350B, may have a sensorarray with the same number of pixels as is provided in the sensor arrayof the larger camera channel, e.g., camera channel 350C, for acomparable portion of the field of view.

In one embodiment, for example, the pixels in the second camera channel,e.g., camera channel 350B, are forty four percent larger in size (e.g.,twenty percent larger in the x direction and twenty percent larger inthe y direction) than the pixels in the smallest camera channel, e.g.,camera channel 350A. In another embodiment, for example, the pixels inthe largest camera channel, e.g., camera channel 350C, are thirty sixpercent larger in size (e.g., seventeen percent larger in the xdirection and seventeen percent larger in the y direction) than thepixels in the second camera channel, e.g., camera channel 350B. Itshould be understood, however, that any other sizes and/or architecturesmay also be employed.

In some embodiments, the first camera channel, e.g., camera channel350A, is dedicated to a first color or first band of colors (e.g., redor a red band of colors), the second camera channel, e.g., camerachannel 350B, is dedicated to second color or second band of colors(e.g., blue or a blue band of colors), different than the first color orfirst band of colors, and the third camera channel, e.g., camera channel350C, is dedicated to third color or third band of colors (e.g., greenor a green band of colors), different than the first and second colorsor band of colors. In some of such embodiments, the two smaller camerachannels, e.g., camera channels 350A-350B, each have a lower resolutionthan the third camera channel, e.g., camera channel 350C. In some otherembodiments, each of the camera channels, e.g., camera channels350A-350C, has the same resolution.

In some embodiments, the number of pixels and/or the design of thepixels in the sensor array of a camera channel are adapted to match awavelength or band of wavelengths of incident light to which such camerachannels may be dedicated.

In some embodiments, the size of the sensor array and/or the design ofthe optics (e.g., f/# and focal length) for one or more of the camerachannels are adapted to provide a desired field of view and/orsensitivity for such camera channels.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, one or more of the camera channels are dedicated toa wavelength or band of wavelengths and the sensor array and/or opticsportion of such one or more camera channels are optimized for therespective wavelength or band of wavelengths to which the respectivecamera channel is dedicated. In some embodiments, the design, operation,array size and/or pixel size of each sensor array is optimized for therespective wavelength or bands of wavelengths to which the camerachannels are dedicated. In some embodiments the design of each opticalportion is optimized for the respective wavelength or bands ofwavelengths to which the respective camera channel is dedicated.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, FIGS. 100B-100D. In some embodiments, the processor may haveone or more portions that are not disposed on the same integratedcircuit as the sensor arrays and/or may not have any portions disposedon the same integrated circuit as the sensor arrays (see for example,FIG. 10A).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, the camera channels, e.g., camera channels350A-350C, are connected via one or more communication links to one ormore displays. In some such embodiments, each of the camera channels isconnected to its own display. The displays may or may not have the samecharacteristics. For example, in some embodiments, one or more of thecamera channels have a resolution that is less than the resolution ofone or more other camera channels and the one or more display connectedto the one or more lower resolution camera channels may have aresolution/resolutions that is/are lower than the resolution/resolutionsof the one or more displays connected to the one or more other camerachannels.

Three Elliptical Channels of all Different Sizes

FIGS. 103A-103E are schematic representations of digital cameraapparatus 300 in accordance with further embodiments of the presentinvention. In each of these embodiments, the digital camera apparatusinclude one or more camera channels, e.g., camera channels 350A-350C,having an optical portion, e.g., optical portions 330A-330C,respectively, with an elliptical or other type of non circular shape.

In some embodiments, one or more of the camera channels, e.g., camerachannels 350A-350B, are smaller in size than a third camera channel,e.g., camera channel 350C. In some of such embodiments, the one or moresmaller camera channels, e.g., camera channels 350A-350B, may each havea resolution that is less than the resolution of the larger camerachannel, e.g., camera channel 350C, although in such embodiments, thesmaller camera channels may or may not have the same resolution as oneanother. For example, each of the smaller camera channels, e.g., camerachannel 350A-350B, may have a sensor array with fewer pixels than isprovided in the sensor array of the larger camera channel, e.g., camerachannel 350B, for a comparable portion of the field of view. In oneembodiment, for example, the number of pixels in the larger camerachannel is forty four percent greater than the number of pixels in oneor more of the smaller camera channels, for a comparable portion of thefield of view. In another embodiment, for example, the number of pixelsin the larger camera channel is thirty six percent greater than thenumber of pixels in one or more of the smaller camera channels, for acomparable portion of the field of view. It should be understood,however, that any other sizes and/or architectures may also be employed.

If one or more of the camera channels, e.g., camera channels 350A-350B,are smaller in size than a third camera channel, e.g., camera channel350C, the one or more of the smaller camera channels, may have aresolution that is equal to the resolution of the larger camera channel.For example, one or more of the smaller camera channels, e.g., camerachannels 350A-350B, may have a sensor array with the same number ofpixels as is provided in the sensor array of the larger camera channel,e.g., camera channel 350C, for a comparable portion of the field ofview. In one embodiment, for example, the pixels in the larger camerachannel are forty four percent larger in size (e.g., twenty percentlarger in the x direction and twenty percent larger in the y direction)than the pixels in the smaller camera channels. In another embodiment,for example, the pixels in the larger camera channel are thirty sixpercent larger in size (e.g., seventeen percent larger in the xdirection and seventeen percent larger in the y direction) than thepixels in the smaller camera channels. It should be understood, however,that any other sizes and/or architectures may also be employed.

In some embodiments, the first camera channel, e.g., camera channel350A, is dedicated to a first color or first band of colors (e.g., redor a red band of colors), the second camera channel, e.g., camerachannel 350B, is dedicated to second color or second band of colors(e.g., blue or a blue band of colors), different than the first color orfirst band of colors, and the third camera channel, e.g., camera channel350C, is dedicated to third color or third band of colors (e.g., greenor a green band of colors), different than the first and second colorsor band of colors. In some of such embodiments, the two smaller camerachannels, e.g., camera channels 350A-350B, each have a lower resolutionthan the third camera channel, e.g., camera channel 350C.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, FIGS. 103B-103E). In some embodiments, the processor may haveone or more portions that are not disposed on the same integratedcircuit as the sensor arrays and/or may not have any portions disposedon the same integrated circuit as the sensor arrays (see for example,FIG. 103A).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, one or more of the camera channels, e.g., camerachannels 350A-350C, are connected via one or more communication links toone or more displays. In some such embodiments, each of the camerachannels is connected to its own display. The displays may or may nothave the same characteristics as one another. For example, if the camerachannels have different resolutions from one another, the displays mayalso have different resolutions from one another. For example, in someembodiments, the smaller channels, e.g., camera channels 350A-350B, havea resolution that is less than the resolution of the larger channel,e.g., camera channel 350B, and the displays connected to the smallerchannels have resolution that is less than the resolution of the displayconnected to the larger camera channel. In some other embodiments, thetwo smaller camera channels, e.g., camera channels 350A-350B, areconnected to a first display and the larger camera channel, e.g., camerachannel 350C is connected to a second display. The first and seconddisplays may or may not have the same characteristics as one another.For example, in some embodiments, the smaller channels, e.g., camerachannels 350A-350B, have a resolution that is less than the resolutionof the larger channel, e.g., camera channel 350B, and the displayconnected to the smaller channels has a resolution that is less than theresolution of the display connected to the larger camera channel.

Two Camera Channels

FIGS. 104A-104E are schematic representations of digital cameraapparatus 300 in accordance with further embodiments of the presentinvention. In each of these embodiments, the digital camera apparatusinclude two or more camera channels, e.g., camera channels 350A-350B.

In some embodiments, the first camera channel, e.g., camera channel350A, is dedicated to a single color or a single band of colors (e.g.,red or a red band) and the second camera channel, e.g., camera channel350B, is dedicated to a single color or single band of colors (e.g.,green or a green band), different from the color or band of colors towhich the first camera channel is dedicated. In some embodiments, one ormore of the camera channels employs a pixel size that matches the coloroptical blur for the respective camera channel, an integration timeand/or other electrical characteristic of the sensor array adapted toincrease or optimize performance of the respective camera channel,and/or a design/layout of the pixel circuitry and photodiode that isadapted to increase or maximize sensitivity of the respective camerachannel.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some other embodiments, the first camera channel, e.g., camerachannel 350A, is dedicated to two or more separate colors or two or moreseparate bands of colors (e.g., blue or a blue band and red or a redband). In some of such embodiments, the optical portion may itself havethe capability to provide color separation, for example, similar to thatprovided by a color filter array (e.g., a Bayer pattern or variationthereof). In some embodiments, the lens and/or filter of the camerachannel may transmit both of such colors or bands of colors and thecamera channel may include one or mechanisms elsewhere in the camerachannel to separate the two colors or two bands of colors. For example,a color filter array may be disposed between the lens and the sensorarray and/or the camera channel may employ a sensor capable ofseparating the colors or bands of colors. In some of the latterembodiments, the sensor array may be provided with pixels that havemultiband capability, e.g., two or three colors. For example, each pixelmay comprise two or three photodiodes, wherein a first photodiode isadapted to detect a first color or first band of colors, a secondphotodiode is adapted to detect a second color or band of colors and athird photodiode is adapted to detect a third color or band of colors.One way to accomplish this is to provide the photodiodes with differentstructures/characteristics that make them selective, such that firstphotodiode has a higher sensitivity to the first color or first band ofcolors than to the second color or band of colors, and the secondphotodiode has a higher sensitivity to the second color or second bandof colors than to the first color or first band of colors. Another wayis to dispose the photodiodes at different depths in the pixel, whichtakes advantage of the different penetration and absorptioncharacteristics of the different colors or bands of colors. For example,blue and blue bands of colors penetrate less (and are thus absorbed at alesser depth) than green and green bands of colors, which in turnpenetrate less (and are thus absorbed at a lesser depth) than red andred bands of colors. In some embodiments, such a sensor array isemployed even though the pixels may see only one particular color orband of colors, for example, to in order to adapt such sensor array tothe particular color or band of colors.

In some of the latter embodiments, the second camera channel, e.g.,camera channel 350B, is dedicated to a single color or a single band ofcolors (e.g., green or a green band), different from the colors or bandof colors to which the first camera channel is dedicated. In some otherembodiments, the second camera channel, e.g., camera channel 350B, isalso dedicated to two or more separate colors or two or more separatebands of colors. For example, the first camera channel may be dedicatedto red or a red band and green or a green band (e.g;, G1). The secondcamera channel may be dedicated to blue or a blue band and green or agreen band (e.g., G2).

The two or more camera channels may or may not have the sameconfiguration (e.g., size, shape, resolution, or degree or range ofsensitivity) as one another. In some embodiments, for example, each ofthe camera channels has the same size, shape, resolution and/or degreeor range of sensitivity as one another. In some other embodiments, oneor more of the camera channels has a size, shape, resolution and/or adegree or range of sensitivity that is different than one or more of theother camera channels. For example, one or more of the camera channelsmay have a sensor array with fewer pixels than is provided in the sensorarray of one or more of the other camera channels, for a comparableportion of the field of view.

In some embodiments, one of the camera channels may have a sensor arraywith a size that is different than the size of the sensor array of theother camera channels. In some of such embodiments, the optics portionof such one or more camera channels may have a f/# and/or a focal lengththat is different from the f/# and/or a focal length of the one or moreof the other camera channels.

In some embodiments, one or more of the camera channels are dedicated toa wavelength or band of wavelengths and the sensor array and/or opticsportion of such one or more camera channels are optimized for therespective wavelength or band of wavelengths to which the respectivecamera channel is dedicated. In some embodiments, the design, operation,array size and/or pixel size of each sensor array is optimized for therespective wavelength or bands of wavelengths to which the camerachannels are dedicated. In some embodiments the design of each opticalportion is optimized for the respective wavelength or bands ofwavelengths to which the respective camera channel is dedicated.

It should be understood however, that any other configurations may beemployed.

The two or more camera may be arranged in any manner. In someembodiments, the two or more camera channels are arranged in lineararray, as shown.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, 104C-104E). In some embodiments, the processor may have one ormore portions that are not disposed on the same integrated circuit asthe sensor arrays and/or may not have any portions integrated in ordisposed on the same integrated circuit as the sensor arrays (see forexample, FIG. 104B).

As stated above, each of the embodiments described above may be usedalone or in combination with any other embodiment(s) or portion thereofdescribed herein or known to those of ordinary skill in the art.

In some embodiments, one or more of the camera channels, e.g., camerachannels 350A-350B, are connected via one or more communication links toone or more displays. In some such embodiments, each of the camerachannels is connected to a separate display, i.e., the smaller camerachannel, e.g., camera channel 350A, is connected to a first display andthe larger camera channel, e.g., camera channel 350B, is connected to asecond display. The first and second displays may or may not have thesame characteristics. In some embodiments, the first display has aresolution equal to the resolution of the camera channel connectedthereto. The second display may have a resolution equal to theresolution of the camera channels connected thereto, however, otherresolutions may be employed. Other resolutions may also be employed.

Two Channels, One Smaller Than Other

FIGS. 105A-105E are schematic representations of digital cameraapparatus 300 in accordance with further embodiments of the presentinvention. In each of these embodiments, the digital camera apparatusinclude two or more camera channels, e.g., camera channels 350A-350B. Afirst one of the camera channels, e.g., camera channel 350A, is smallerin size than a second one of the camera channel, e.g., camera channel350B.

In some embodiments, the smaller camera channel, e.g., camera channel350A, has resolution that is less than the resolution of the largercamera channel, e.g., camera channel 350B. For example, the smallercamera channel, e.g., camera channel 350A, may have a sensor array withfewer pixels than the sensor array of the larger camera channel, e.g.,camera channel 350B, for a comparable portion of the field of view. Inone embodiment, for example, the number of pixels in the larger camerachannel is forty four percent greater than the number of pixels in thesmaller camera channel, for a comparable portion of the field of view.In another embodiment, for example, the number of pixels in the largercamera channel is thirty six percent greater than the number of pixelsin the smaller camera channel, for a comparable portion of the field ofview. It should be understood, however, that any other sizes and/orarchitectures may also be employed.

In some other embodiments, the smaller camera channel, e.g., camerachannel 350A, has a resolution that is equal to the resolution of thelarger camera channel, e.g., camera channel 350B. For example, thesmaller camera channel, e.g., camera channel 350A, may have a sensorarray with pixels that are equal in number but smaller than the pixelsof the sensor array of the larger camera channel, e.g., camera channel350B, for a comparable portion of the field of view. In one embodiment,for example, the pixels in the larger camera channel are forty fourpercent larger in size (e.g., twenty percent larger in the x directionand twenty percent larger in the y direction) than the pixels in thesmaller camera channel. In another embodiment, for example, the pixelsin the larger camera channel are thirty six percent larger in size(e.g., seventeen percent larger in the x direction and seventeen percentlarger in the y direction) than the pixels in the smaller camerachannel. It should be understood, however, that any other sizes and/orarchitectures may also be employed.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

In some embodiments, the first camera channel, i.e., the smaller camerachannel, is dedicated to a single color or a single band of colors(e.g., red or a red band) and the second camera channel, i.e., thelarger camera channel, is dedicated to a single color or single band ofcolors (e.g., green or a green band), different from the color or bandof colors to which the first camera channel is dedicated.

In some other embodiments, one of the camera channels, e.g., the smallercamera channel, is dedicated to two or more separate colors or two ormore separate bands of colors (e.g., blue or a blue band and red or ared band). In some of such embodiments, the lens and/or filter of thecamera channel may transmit both of such colors or bands of colors andthe camera channel may include one or mechanisms elsewhere in the camerachannel to separate the two colors or two bands of colors. For example,a color filter array may be disposed between the lens and the sensorarray and/or the camera channel may employ a sensor capable ofseparating the colors or bands of colors. In some of the latterembodiments, the sensor may be provided with sensor elements or pixelsthat each comprise two photodiodes, wherein the first photodiode isadapted to detect the a first color or first band of colors and thesecond photodiode is adapted to detect the second color or band ofcolors. One way to accomplish this is to provide the photodiodes withdifferent structures/characteristics that make them selective, such thatfirst photodiode has a higher sensitivity to the first color or firstband of colors than to the second color or band of colors, and thesecond photodiode has a higher sensitivity to the second color or secondband of colors than to the first color or first band of colors. Anotherway is to dispose the photodiodes at different depths in the pixel,which takes advantage of the different penetration and absorptioncharacteristics of the different colors or bands of colors. For example,blue and blue bands of colors penetrate less (and are thus absorbed at alesser depth) than green and green bands of colors, which in turnpenetrate less (and are thus absorbed at a lesser depth) than red andred bands of colors.

In some embodiments, the digital camera apparatus employs a processorthat is disposed on the same integrated circuit as the sensor arrays.The processor may have any layout, including for example, a layout thatis the same as, or similar to, the layouts described herein (see forexample, 105C-105E). In some embodiments, the processor may have one ormore portions that are not disposed on the same integrated circuit asthe sensor arrays and/or may not have any portions integrated in ordisposed on the same integrated circuit as the sensor arrays (see forexample, FIG. 105B). As stated above, each of the embodiments describedabove may be used alone or in combination with any other embodiment(s)or portion thereof described herein or known to those of ordinary skillin the art.

To that effect, in some embodiments, the smaller camera channel isdedicated to two or more separate colors (or two or more separate bandsof colors) and in addition, has a resolution that is less than theresolution of the larger camera channel.

In some embodiments, the camera channels, e.g., camera channels350A-350B, are connected via one or more communication links to one ormore displays. In some such embodiments, each of the camera channels isconnected to a separate display, i.e., the smaller camera channel, e.g.,camera channel 350A, is connected to a first display and the largercamera channel, e.g., camera channel 350B, is connected to a seconddisplay, different than the first display. The first and second displaysmay or may not have the same characteristics. In some embodiments, thefirst display has a resolution equal to the resolution of the camerachannel connected thereto. The second display may have a resolutionequal to the resolution of the camera channels connected thereto,however, other resolutions may be employed.

Group of Image Sensor Chips

FIGS. 106A-106C are schematic perspective views of a system having aplurality of digital camera apparatus, e.g., two digital cameraapparatus, in accordance with another embodiment of the presentinvention. The plurality of digital camera apparatus may be arranged inany desired manner. In some embodiments, it may be desired to collectimages from opposing directions. In some embodiments, the digital cameraapparatus are mounted back to back, as shown. Some of such embodimentsmay allow concurrent imaging in opposing directions.

In some embodiments, the one or more optics portions for the firstcamera subsystem face in a direction that is opposite to the directionthat the one or more optics portions for the second digital cameraapparatus face. In the illustrated embodiment, for example, the systemhas first and second sides generally opposite one another. A first oneof the digital camera apparatus may be positioned to receive lightthrough a first side of the digital camera apparatus. A second one ofthe digital camera apparatus may be positioned to receive light throughthe second side of the system. Other configurations may also beemployed.

In some embodiments, each of the subsystems has its own sets of optics,filters and sensors arrays, and may or may not have the sameapplications and/or configurations as one another, for example, in someembodiments, one of the subsystems may be a color system and the othermay be a monochromatic system, one of the subsystems may have a firstfield of view and the other may have a separate field of view, one ofthe subsystems may provide video imaging and the other may provide stillimaging.

The plurality of digital camera subassemblies may have any size andshape and may or may not have the same configuration as one another(e.g., type, size, shape, resolution). In the illustrated embodiment,one of the subsystems has a length and a width that are equal to thelength and width, respectively of the other subsystem, although this isnot required.

In some embodiments, one or more sensor portions for the second digitalcamera apparatus are disposed on the same device (e.g., image device) asone or more sensor portions for the first digital camera apparatus. Insome embodiments, one or more sensor portions for the second digitalcamera apparatus are disposed on a second device (e.g., a second imagedevice), which may be disposed, for example, adjacent to the imagedevice on which the one or more sensor portions for the first digitalcamera apparatus are disposed.

In some embodiments, two or more of the digital camera apparatus share aprocessor, or a portion thereof. In some other embodiments, each of thedigital camera apparatus has its own dedicated processor separate fromthe processor for the other digital camera apparatus.

In some embodiments, the system defines a hermetic package, althoughthis is not required.

As with each of the embodiments disclosed herein, this embodiment of thepresent invention may be employed alone or in combination with one ormore of the other embodiments disclosed herein, or portion thereof.

The digital camera apparatus may be assembled and/or mounted in anymanner, for example, but not limited to in a manner similar to thatemployed in one or more of the embodiments disclosed herein.

FIGS. 107A-107B are a schematic representation of another embodiment.This embodiment includes a plurality of image devices. In thisembodiment each of the each image devices has one or more sensor arraysfor one or more camera channels. In some embodiments, the image devicesmay or may not be similar to one another.

Although many of the Figures herein show a digital camera subassembly inthe form of a layered assembly, it should be understood that the digitalcamera subassembly may or may not have such a configuration. Indeed, theone or more camera channels of a digital camera subassembly may have anyconfiguration. Thus, some embodiments may have the form of a layeredassembly. Some other embodiments may not have the form of a layeredassembly.

For example, FIGS. 108A-108B are a schematic representation of digitalcamera sub assemblies in accordance with further embodiments of thepresent invention. The digital camera subassemblies each employ one ormore of the embodiments described herein, or portions thereof. However,the digital camera subassemblies may or may not have the form of alayered assembly.

In this embodiment, the digital camera assembly includes one or morecamera channels. The camera channels may have any configurations and mayor may not have the same configuration as one another.

In some embodiments, each of the camera channels comprises a 2 M pixelnarrow band camera, e.g., a red camera channel, a blue camera channeland a green camera channel.

In some embodiments, each of the camera channels comprises a 1.3 M pixelnarrow band camera, e.g., a red camera channel, a blue camera channeland a green camera channel.

In some embodiments, one of the camera channels is a broadband camerachannel, an infrared (IR) camera channel or an ultraviolet (UV) camerachannel.

However, as stated above, other configurations may also be employed.

As with each of the embodiments disclosed herein, this embodiment of thepresent invention may be employed alone or in combination with one ormore of the other embodiments disclosed herein, or portion thereof.

Active Support

In some embodiments, the position of each optics portion relative to therespective sensor portion is fixed. In some alternative embodimentshowever, one or more actuators may be provided to provide movement ofone or more of the optics portions, or portions thereof, and/or one ormore sensor arrays, or portions thereof. In some embodiments, one ormore of such actuators are provided in the support (such a support maycomprise, for example, a frame provided with one or more actuators).

For example, it may be desirable to provide relative movement between anoptics portion (or one or more portions thereof) and a sensor array (orone or more portions thereof), including, for example, but not limitedto relative movement in the x and/or y direction, z direction, tilting,rotation (e.g., rotation of less than, greater than and/or equal to 360degrees) and/or combinations thereof, may be used in providing variousfeatures and/or in the various applications disclosed herein, including,for example, but not limited to, increasing resolution (e.g., increasingdetail), zoom, 3D enhancement, image stabilization, image alignment,lens alignment, masking, image discrimination, auto focus, mechanicalshutter, mechanical iris, hyperspectral imaging, a snapshot mode, rangefinding and/or combinations thereof.

Notably, the inventions described and illustrated in the U.S.Provisional Application Ser. No. 60/695,946, entitled “Method andApparatus for use in Camera and Systems Employing Same”, filed Jul. 1,2005, may be employed in conjunction with the present inventions. Forthe sake of brevity, those discussions will not be repeated. It isexpressly noted that the entire contents of the aforementioned U.S.Provisional Application, including, for example, the features,attributes, alternatives, materials, techniques and/or advantages of allof the inventions/embodiments thereof, are incorporated by referenceherein.

Such movement may be provided for example using actuators, e.g., MEMSactuators and by applying appropriate control signal(s) to one or moreof the actuators to cause the one or more actuators to move, expandand/or contract to thereby move the associated optics portion. It may beadvantageous to make the amount of movement equal to a small distance,e.g., 2 microns (2 um), which may be sufficient for many applications.In some embodiments, for example, the amount of movement may be as smallas about ½ of the width of one sensor element (e.g., ½ of the width ofone pixel) on one of the sensor arrays. In some embodiments, forexample, the magnitude of movement may be equal to the magnitude of thewidth of one sensor element or two times the magnitude of the width ofone sensor element.

In some embodiments, the relative movement is in the form of a ⅓ pixel×⅓pixel pitch shift in a 3×3 format. In other embodiments, the relativemovement is in the form of dithering. In some dithered systems, it maybe desirable to employ a reduced optical fill factor. In someembodiments, snap-shot integration is employed. Some embodiments providethe capability to read out a signal while integrating.

In some embodiments, the digital camera apparatus employs relativemovement by itself, in lieu of or in combination with one or moreembodiments disclosed herein in providing various features and/or in thevarious applications disclosed herein, for example, but not limited to,increasing resolution (e.g., increasing detail), zoom, 3D effects, imagestabilization, image alignment, lens alignment, masking, imagediscrimination, auto focus, auto exposure, mechanical shutter,mechanical iris, hyperspectral imaging, snapshot mode, range findingand/or combinations thereof.

In addition, it should also be understood that such relative movementmay be employed in providing any other features and/or in variousapplications now known or later developed, and may also be employed, ifdesired, with any methods and/or apparatus, now known or laterdeveloped.

FIGS. 109A-109D are block diagram representation showing configurationsemployed in some embodiments of the present invention.

Although some of the Figures herein show the processor separate from thesensor arrays, the processor, or portions thereof, may have anyconfiguration and may be disposed in any location or locations. In someembodiments, one, some or all portions of the processor are disposed onthe same substrate or substrates as one or more of the one or more ofthe sensor arrays, e.g., sensor arrays 310A-310D. However, in someembodiments one, some or all portions of the processor are disposed onone or more substrates that are separate from (and possibly remote from)one or more substrates on which one or more of the one or more sensorarrays, e.g., sensor arrays 310A-310D, may be disposed.

In some embodiments, one or more portions of the digital cameraapparatus include circuitry to facilitate wired, wireless and/or opticalcommunication to and/or from the subsystem and/or within the subsystem.Such circuitry may have any form. In some embodiments, one or moreportions of such circuitry may be part of the processor 340 and may bedisposed in the same integrated circuit as one or more other portions ofthe processor 340 and/or may be in a discrete form, separate from theprocessor 340 or other portions thereof.

FIG. 110A is a block diagram representation of the processor 340 inaccordance with one embodiment of aspects of the present invention. Inthis embodiment, the processor 340 includes one or more channelprocessors, one or more image pipelines, and/or one or more image postprocessors. Each of the channel processors is coupled to a respectiveone of the camera channels and generates an image based at least in parton the signal(s) received from the respective camera channel. In someembodiments the processor 340 generates a combined imaged based at leastin part on the images from two or more of the camera channels. In someembodiments, one or more of the channel processors are tailored to itsrespective camera channel, for example, as described herein. Forexample, if one of the camera channels is dedicated to a specificwavelength or color (or band of wavelengths or colors), the respectivechannel processor may also be adapted to such wavelength or color (orband of wavelengths or colors). Any of the other embodiments describedherein or combinations thereof, may also be employed.

For example, the gain, noise reduction, dynamic range, linearity and/orany other characteristic of the processor, or combinations of suchcharacteristics, may be adapted to improve and/or optimize the processorto such wavelength or color (or band of wavelengths or colors).Tailoring the channel processing to the respective camera channel mayhelp to make it possible to generate an image of a quality that ishigher than the quality of images resulting from traditional imagesensors of like pixel count. In such embodiments, providing each camerachannel with a dedicated channel processor may help to reduce orsimplify the amount of logic in the channel processors as the channelprocessor may not need to accommodate extreme shifts in color orwavelength, e.g., from a color (or band of colors) or wavelength (orband of wavelengths) at one extreme to a color (or band of colors) orwavelength (or band of wavelengths) at another extreme

The images (and/or data which is representative thereof) generated bythe channel processors are supplied to the image pipeline, which maycombine the images to form a full color or black/white image. The outputof the image pipeline is supplied to the post processor, which generatesoutput data in accordance with one or more output formats.

FIG. 110B shows one embodiment of a channel processor. In thisembodiment, the channel processor includes column logic, analog signallogic, black level control and exposure control. The column logic iscoupled to the sensor and reads the signals from the pixels. If thechannel processor is coupled to a camera channel that is dedicated to aspecific wavelength (or band of wavelengths), it may be advantageous forthe column logic to be adapted to such wavelength (or band ofwavelengths). For example, the column logic may employ an integrationtime or integration times adapted to provide a particular dynamic rangein response to the wavelength (or band of wavelengths) to which thecolor channel is dedicated. Thus, it may be advantageous for the columnlogic in one of the channel processors to employ an integration time ortimes that is different than the integration time or times employed bythe column logic in one or more of the other channel processors.

The analog signal logic receives the output from the column logic. Ifthe channel processor is coupled to a camera channel dedicated to aspecific wavelength or color (or band of wavelengths or colors), it maybe advantageous for the analog signal logic to be specifically adaptedto such wavelength or color (or band of wavelengths or colors). As such,the analog signal logic can be optimized, if desired, for gain, noise,dynamic range and/or linearity, etc. For example, if the camera channelis dedicated to a specific wavelength or color (or band of wavelengthsor colors), dramatic shifts in the logic and settling time may not berequired as each of the sensor elements in the camera channel arededicated to the same wavelength or color (or band of wavelengths orcolors). By contrast, such optimization may not be possible if thecamera channel must handle all wavelength and colors and employs a Bayerarrangement in which adjacent sensor elements are dedicated to differentcolors, e.g., red-blue, red-green or blue-green.

The output of the analog signal logic is supplied to the black levellogic, which determines the level of noise within the signal, andfilters out some or all of such noise. If the sensor coupled to thechannel processor is focused upon a narrower band of visible spectrumthan traditional image sensors, the black level logic can be more finelytuned to eliminate noise. If the channel processor is coupled to acamera channel that is dedicated to a specific wavelength or color (orband of wavelengths or colors), it may be advantageous for the analogsignal logic to be specifically adapted to such wavelength or color (orband of wavelengths or colors).

The output of the black level logic is supplied to the exposure control,which measures the overall volume of light being captured by the arrayand adjusts the capture time for image quality. Traditional cameras mustmake this determination on a global basis (for all colors). If thesensor coupled to the channel processor is dedicated to a specific color(or band of colors, the exposure control can be specifically adapted tothe wavelength (or band of wavelengths) to which the sensor is targeted.Each channel processor is thus able to provide a capture time that isspecifically adapted to the sensor and/or specific color (or band ofcolors) targeted thereby and different than the capture time provided byone or more of the other channel processors for one or more of the othercamera channels.

FIG. 110C shows one embodiment of the image pipeline. In thisembodiment, the image pipeline includes two portions. The first portionincludes a color plane integrator and an image adjustor. The color planeintegrator receives an output from each of the channel processors andintegrates the multiple color planes into a single color image. Theoutput of the color plane integrator, which is indicative of the singlecolor image, is supplied to the image adjustor, which adjusts the singlecolor image for saturation, sharpness, intensity and hue. The adjustoralso adjusts the image to remove artifacts and any undesired effectsrelated to bad pixels in the one or more color channels. The output ofthe image adjustor is supplied to the second portion of the pipeline,which provides auto focus, zoom, windowing, pixel binning and camerafunctions.

FIG. 110D shows one embodiment of the image post processor. In thisembodiment, the image post processor includes an encoder and an outputinterface. The encoder receives the output signal from the imagepipeline and provides encoding to supply an output signal in accordancewith one or more standard protocols (e.g., MPEG and/or JPEG).

The output of the encoder is supplied to the output interface, whichprovides encoding to supply an output signal in accordance with astandard output interface, e.g., universal serial bus (USB) interface.

FIG. 110E shows one embodiment of the system control. In thisembodiment, the system control portion includes a serial interface,configuration registers, power management, voltage regulation andcontrol, timing and control, a camera control interface and a serialinterface. In some embodiments, the camera interface comprises aninterface that processes signals that are in the form of high levellanguage (HLL) instructions. The following paragraphs describe one suchembodiment of the system control. It should be understood however, thatthe camera interface is not limited to such an embodiment and may haveany configuration. In some embodiments the camera interface comprises aninterface that processes control signals that are in the form of lowlevel language (LLL) instructions and/or of any other form now known orlater developed. Some embodiments may process both HLL instructions andLLL instructions.

In the operation of this embodiment, communication occurs through theserial interface which is connected to a serial port. For example,signals indicative of instructions (e.g., HLL camera controlinstructions), desired settings, operations and/or data are supplied tothe serial interface and control portion through the through the serialport. If the signals are not indicative of a HLL camera controlinstruction (i.e., a HLL instruction having to do with the camera),signals indicative of the desired settings, operations and/or data aresupplied to the configuration registers to be stored therein. If thesignals are indicative of a HLL camera control instruction, then the HLLinstruction is supplied to the HLL camera control interface. The HLLcamera control interface decodes the instruction to generate signalsindicative of desired (by the user or other device) settings, operationsand/or data, which are supplied to the configuration registers to bestored therein.

Signals indicative of the desired settings, operations and/or data aresupplied to the power management, sensor timing and control portion,channel processors, image pipeline and image post processor, asappropriate. The power management portion receives the signals suppliedand, in response at least thereto, supplies control signals to thevoltage regulation power and control portion, which in turn connects tocircuits in the digital camera apparatus. The sensor timing and controlportion receives the signals supplied and, in response at least thereto,supplies control signals to the sensor arrays to control the operationthereof. The channel processors receive the signals supplied through (orlines), and further receives one or more signals from one or more of thesensor arrays and performs, in response at least thereto, one or morechannel processor operations.

The image pipeline receives the signals and further receives one or moresignals from one or more of the channel processors and performs, inresponse at least thereto, one or more image pipeline operations.

The image post processor receives the signals and further receives oneor more signals from the image pipeline and performs, in response atleast thereto, one or more image post processor operations.

FIG. 110F shows an example of a high level language camera controlinstruction, according to one embodiment of the present invention. Theinstruction format has an op code, e.g., COMBINE, which in this caseidentifies the instruction as a type of camera control instruction andrequests that the digital camera apparatus generate a combined image.The instruction format also has one or more operands fields, e.g.,channel id1, channel id2, which identify the camera channels to be usedin generating the combined image, at least in part.

As used herein, the term “combined image” means an image based, at leastin part, on information captured by two or more camera channels. Thecombined image may be generated in any manner. Example camera channelsinclude but are not limited to camera channels 350A-350D.

An example of a HLL camera control instruction that uses the instructionformat of FIG. 110E is “COMBINE 1, 2””—this instruction calls for anoutput image based at least in part on information captured by a camerachannel designated as “camera channel 1”, e.g., camera channel 350A, anda camera channel designated as “camera channel 2”, e.g., camera channel350B.

Another example is COMBINE 1, 2, 3, 4—this instruction calls for anoutput image based at least in part on information captured by a camerachannel designated as “camera channel 1”, e.g., camera channel 350A, anda camera channel designated as “camera channel 2”, e.g., camera channel350B, a camera channel designated as “camera channel 3”, e.g., camerachannel 350C, and a camera channel designated as “camera channel 4”,e.g., camera channel 350D.

The availability of a COMBINE and other HLL instruction for the digitalcamera apparatus provides instructions in a form that is closer to humanlanguage than is the form of machine language and/or assembly language,thereby helping to bring about writing, reading and/or maintainingprograms for the digital camera apparatus.

It should be recognized that the present invention is not limited to theCOMBINE instruction and HLL instruction format shown in FIG. 110E andthat other instruction formats including for example other HLLinstruction formats may be used.

For example, in some embodiments, the camera channels are not specifiedin the instruction, but rather are implied, for example, based on the opcode. In such embodiments, the digital camera apparatus may beconfigured, for example, to automatically generate a combined imagebased at least in part on a group of predetermined camera channelswhenever a COMBINE instruction is supplied. Alternatively, for example,a plurality of different COMBINE or other HLL instructions may besupported, each having a different op code. The different op codes mayimplicitly identify the particular camera channels of interest. Forexample, the instruction “COMBINE12” may call for an output image basedat least in part on information captured by a camera channel designatedas “camera channel 1”, e.g., camera channel 350A, and a camera channeldesignated as “camera channel 2”, e.g., camera channel 350B.

The instruction “COMBINE1234” may call for an output image based atleast in part on information captured by a camera channel designated as“camera channel 1”, e.g., camera channel 350A, and a camera channeldesignated as “camera channel 2”, e.g., camera channel 350B, a camerachannel designated as “camera channel 3”, e.g., camera channel 350C, anda camera channel designated as “camera channel 4”, e.g., camera channel350D.

In some embodiments, a single COMBINE instruction causes more than onecombined image to be generated. The camera channels of interest for theadditional(s) combined images may be implied based on the opcode (e.g.,as discussed above). Alternatively, for example the camera channels ofinterest for the additional combined images may be implied based on thesupplied operands.

FIG. 110G shows high level language instructions in accordance withfurther embodiments of the present invention.

In some embodiments, one or more of the instructions may cause thecamera interface to initiate the operation suggested thereby. Forexample, if the instruction “Whte Balance Manual” is received, thecamera interface may instruct the white balance control to operate inmanual mode and/or initiate signals that eventually cause the camera tooperate in such a mode.

These instructions may be used, for example, to control the cameraset-up and/or the operating mode of one or more aspects of the camerathat have two or more states, e.g., “on/off” and/or “manual/auto”.

Some embodiments include one, some or all of the instructions of FIG.110G. Some other embodiments may not employ any of the instructionslisted in FIG. 110G.

FIG. 110H shows high level language instructions in accordance withfurther embodiments of the present invention.

In some embodiments, one or more of the instructions may cause thecamera interface to initiate the operation suggested thereby. Forexample, if the instruction “Single Frame Capture” is received, thecamera interface may initiate capture of a single frame.

Some embodiments include one, some or all of the instructions of FIG.110H. Some other embodiments may not employ any of the instructionslisted in FIG. 110H. Some embodiments may include one or moreinstructions from FIG. 110G and one or more instructions from FIG. 110H,alone and/or in combination with signals of any other form.

In some embodiments, the camera interface can be configured to providelimited access to Low Level Commands to provide specific user definedfunctionality.

The form of the signals for the camera interface may be predetermined,adaptively determined and/or user determined. In some embodiments, forexample, a user may define an instruction set and/or a format for theinterface.

As stated above, the camera interface is not limited to embodiments thatemploy a HLL camera control interface. The camera interface may have anyconfiguration. In some embodiments the camera interface comprises aninterface that processes control signals that are in the form of lowlevel language (LLL) instructions and/or of any other form now known orlater developed. Some embodiments may process both HLL instructions andLLL instructions.

It should be understood that the processor 340 is not limited to theportions and/or operations set forth above. For example, the processor340 may comprise any type of portions or combination thereof and/or maycarry out any operation or operations.

It should also be understood that the processor 340 may be implementedin any manner. For example, the processor 340 may be programmable or nonprogrammable, general purpose or special purpose, dedicated or nondedicated, distributed or non distributed, shared or not shared, and/orany combination thereof. If the processor 340 has two or moredistributed portions, the two or more portions may communicate via oneor more communication links. A processor may include, for example, butis not limited to, hardware, software, firmware, hardwired circuitsand/or any combination thereof. In some embodiments, one or moreportions of the processor 340 may be implemented in the form of one ormore ASICs. The processor 340 may or may not execute one or morecomputer programs that have one or more subroutines, or modules, each ofwhich may include a plurality of instructions, and may or may notperform tasks in addition to those described herein. If a computerprogram includes more than one module, the modules may be parts of onecomputer program, or may be parts of separate computer programs. As usedherein, the term module is not limited to a subroutine but rather mayinclude, for example, hardware, software, firmware, hardwired circuitsand/or any combination thereof.

In some embodiments, the processor 340 includes circuitry to facilitatewired, wireless and/or optical communication to and/or from the digitalcamera apparatus. Such circuitry may have any form. In some embodiments,one or more portions of such circuitry is disposed in the sameintegrated circuit as the other portions of the processor 340. In someembodiments, one or more portions of such circuitry are in a discreteform, separate from the integrated circuit for the other portions of theprocessor 340 or portions thereof.

In some embodiments, the processor 340 comprises at least one processingunit connected to a memory system via an interconnection mechanism(e.g., a data bus). A memory system may include a computer-readable andwriteable recording medium. The medium may or may not be non-volatile.Examples of non-volatile medium include, but are not limited to,magnetic disk, magnetic tape, non-volatile optical media andnon-volatile integrated circuits (e.g., read only memory and flashmemory). A disk may be removable, e.g., known as a floppy disk, orpermanent, e.g., known as a hard drive. Examples of volatile memoryinclude but are not limited to random access memory, e.g., dynamicrandom access memory (DRAM) or static random access memory (SRAM), whichmay or may not be of a type that uses one or more integrated circuits tostore information.

If the processor 340 executes one or more computer programs, the one ormore computer programs may be implemented as a computer program producttangibly embodied in a machine-readable storage medium or device forexecution by a computer. Further, if the processor 340 is a computer,such computer is not limited to a particular computer platform,particular processor, or programming language. Computer programminglanguages may include but are not limited to procedural programminglanguages, object oriented programming languages, and combinationsthereof.

A computer may or may not execute a program called an operating system,which may or may not control the execution of other computer programsand provides scheduling, debugging, input/output control, accounting,compilation, storage assignment, data management, communication control,and/or related services. A computer may for example be programmableusing a computer language such as C, C++, Java or other language, suchas a scripting language or even assembly language. The computer systemmay also be specially programmed, special purpose hardware, or anapplication specific integrated circuit (ASIC).

Example output devices include, but are not limited to, displays(e.g.,cathode ray tube (CRT) devices, liquid crystal displays (LCD), plasmadisplays and other video output devices), printers, communicationdevices for example modems, storage devices such as a disk or tape andaudio output, and devices that produce output on light transmittingfilms or similar substrates. An output device may include one or moreinterfaces to facilitate communication with the output device. Theinterface may be any type of interface, e.g., proprietary or notproprietary, standard (for example, universal serial bus (USB) or microUSB) or custom or any combination thereof.

Example input devices include but are not limited to buttons, knobs,switches, keyboards, keypads, track ball, mouse, pen and tablet, lightpen, touch screens, and data input devices such as audio and videocapture devices. An output device may include one or more interfaces tofacilitate communication with the output device. The interface may beany type of interface, for example, but not limited to, proprietary ornot proprietary, standard (for example, universal serial bus (USB) ormicro USB) or custom or any combination thereof.

In addition, as stated above, it should be understood that the featuresdisclosed herein can be used in any combination.

Input signals to the processor 340 may have any form and may be suppliedfrom any source, for example, but not limited to, one or more sourceswithin the digital camera apparatus (e.g., the user peripheral interfaceon the digital camera) and/or one or more other devices. For example, insome embodiments, the peripheral user interface includes one or moreinput devices by which a user can indicate one or more preferences inregard to one or more desired operating modes (e.g., resolution, manualexposure control) and the peripheral user interface generates one ormore signals indicative of such preference or preferences. In someembodiments, one or more portions of the processor 340 generates one ormore signals indicative of one or more desired operating mode. In someembodiments, the one or more portions of the processor 340 generates oneor more of such signals in response to one or more inputs from theperipheral user interface.

In some embodiments, one or more portions of the digital cameraapparatus include circuitry to facilitate wired, wireless and/or opticalcommunication to and/or from the subsystem and/or within the subsystem.Such circuitry may have any form. In some embodiments, one or moreportions of such circuitry may be part of the processor 340 and may bedisposed in the same integrated circuit as one or more other portions ofthe processor 340 and/or may be in a discrete form, separate from theprocessor 340 or other portions thereof.

In some embodiments, the digital camera apparatus includes a memorysection that is supplied with and/or stores one, some or all of theimages and/or other information generated or used by the digital cameraapparatus and/or or any other information from any source and desired tobe stored for any duration. The memory section may supply one or more ofsuch images and/or such other information to one or more other devicesand/or to one or more portions of the processor 340, for example, to befurther processed and/or to be supplied to one or more other devices.The memory section may be, for example, part of the processor 340 and/orcoupled to one or more portions of the processor 340 via one or morecommunication links. In some embodiments, the memory section is alsocoupled to one or more other devices via one or more communicationlinks. In such embodiments, the memory section may supply one or more ofthe stored images and/or other information to one or more of the one ormore other devices, directly (i.e., without passing through the anyother portion of the processor 340) via one or more of the one or morecommunication links, although this is not required.

FIG. 111A shows another embodiment of the channel processor. In thisembodiment, the channel processor includes a double sampler, an analogto digital converter, a black level clamp and a deviant pixelcorrection.

An image may be represented as a plurality of picture element (pixel)magnitudes. Each pixel magnitude indicates the picture intensity(relative darkness or relative lightness) at an associated location ofthe image. A relatively low pixel magnitude indicates a relatively lowpicture intensity (i.e., relatively dark location). Conversely, arelatively high pixel magnitude indicates a relatively high pictureintensity (i.e., relatively light location). The pixel magnitudes areselected from a range that depends on the resolution of the sensor.

FIG. 111B is a graphical representation of a neighborhood of pixelvalues. FIG. 111B further illustrates a plurality of prescribed spatialdirections, namely, a first prescribed spatial direction (e.g., thehorizontal direction), a second prescribed spatial direction (e.g., thevertical direction), a third prescribed spatial direction (e.g., a firstdiagonal direction), and a fourth prescribed spatial direction (e.g., asecond diagonal direction). The pixel P22 is adjacent to pixels P12,P21, P32 and P23. The pixel P22 is offset in the horizontal directionfrom the pixel P32. The pixel P22 is offset in the vertical directionfrom the pixel P23. The pixel P22 is offset in the first diagonaldirection from the pixel P11. The pixel P22 is offset in the seconddiagonal direction from the pixel P31.

The double sampler determines the amount by which the value of eachpixel changes during the exposure period, thereby in effect providing anestimate of the amount of light received by each pixel during anexposure period. For example, a pixel may have a first value, Vstart,prior to an exposure period. The first value, Vstart, may or may not beequal to zero. The same pixel may have a second value, Vend, after theexposure period. The difference between the first and second values,i.e., Vend-Vstart, is indicative of the amount of light received by thepixel.

FIG. 111C shows a flowchart of operations employed in this embodiment ofdouble sampling.

The value of a plurality of pixels in a sensor array are reset to aninitial state prior to the beginning of an exposure period. The value ofeach pixel is sampled prior to the start of an exposure period. Thevalue of each pixel is sampled after the exposure period and signalsindicative thereof are supplied to the double sampler. The doublesampler generates a signal for each pixel, indicative of the differencebetween the start and end values for such pixel.

As stated above, the magnitude of each difference signal is indicativeof the amount of light received at a respective location of the sensorarray. A difference signal with a relatively low magnitude indicatesthat a relatively low amount of light is received at the respectivelocation of the sensor array. A difference signal with a relatively highmagnitude indicates that a relatively high amount of light is receivedat the respective location of the sensor array.

Referring again to FIG. 111A, the difference signals generated by thedouble sampler are supplied to an analog to digital converter, whichsamples each of such signals and generates a sequence of multi-bitdigital signals in response thereto, each multi-bit digital signal beingindicative of a respective one of the difference signals.

The multi-bit digital signals are supplied to a black level clamp, whichcompensates for drift in the sensor array of the camera channel.

The difference signals should have a magnitude equal to zero unless thepixels are exposed to light. However, due to imperfection in the sensor(e.g., leakage currents) the value of the pixels may change (e.g.,increase) even without exposure to light. For example, a pixel may havea first value, Vstart, prior to an exposure period. The same pixel mayhave a second value, Vend, after the exposure period. If drift ispresent, the second value may not be equal to the first value, even ifthe pixel was not exposed to light. The black level clamp compensatesfor such drift.

To accomplish this, in some embodiments, a permanent cover is appliedover one or more portions (e.g., one or more rows and/or one or morecolumns) of the sensor array to prevent light from reaching suchportions. The cover is applied, for example, during manufacture of thesensor array. The difference signals for the pixels in the coveredportion(s) can be used in estimating the magnitude (and direction) ofthe drift in the sensor array.

In this embodiment, the black level clamp generates a reference value(which represents an estimate of the drift within the sensor array)having a magnitude equal to the average of the difference signals forthe pixels in the covered portion(s). The black level clamp thereaftercompensates for the estimated drift by generating a compensateddifference signal for each of the pixels in the uncovered portions, eachcompensated difference signal having a magnitude equal to the magnitudeof the respective uncompensated difference signal reduced by themagnitude of the reference value (which as stated above, represents anestimate of the drift).

The output of the black level clamp is supplied to the deviant pixelidentifier, which seeks to identify defective pixels and help reduce theeffects thereof.

In this embodiment, a defective pixel is defined as pixel for which oneor more values, difference signal and/or compensated difference signalfails to meet one or more criteria, in which case one or more actionsare then taken to help reduce the effects of such pixel. In thisembodiment, for example, a pixel is defective if the magnitude of thecompensated difference signal for the pixel is outside of a range ofreference values (i.e., less than a first reference value or greaterthan a second reference value). The range of reference values may be apredetermined, adaptively determined and/or any combination thereof.

If the magnitude of the compensated difference signal is outside suchrange, then the magnitude of the compensated difference signal is setequal to a value that is based, at least in part, on the compensateddifference signals for one or more pixels adjacent to the defectivepixel, for example, an average of the pixel offset in the positive xdirection and the pixel offset in the negative x direction.

FIG. 111D shows a flowchart of operations employed in this embodiment ofthe defective pixel identifier. The magnitude of each compensateddifference signal is compared to a range of reference values. If amagnitude of a compensated difference signal is outside of the range ofreference values, then the pixel is defective and the magnitude ofdifference signal is set to a value in accordance with the methodologyset forth above.

FIG. 111E shows another embodiment of the image pipeline.

In this embodiment, the image pipeline includes an image planeintegrator, image plane alignment and stitching, exposure control, focuscontrol, zoom control, gamma correction, color correction, edgeenhancement, chroma noise reduction, white balance, color enhancement,image scaling and color space conversion.

The output of a channel processor is a data set that represents acompensated version of the image captured by the camera channel. Thedata set may be output as a data stream. For example, the output fromthe channel processor for camera channel A represents a compensatedversion of the image captured by camera channel A and may be in the formof a data stream P_(A1), P_(A2), . . . P_(An). The output from thechannel processor for camera channel B represents a compensated versionof the image captured by camera channel B and may be in the form of adata stream P_(B1), P_(B2), . . . P_(Bn). The output from the channelprocessor for camera channel C represents a compensated version of theimage captured by camera channel C and is in the form of a data streamP_(C1), P_(C2), . . . P_(Cn). The output from the channel processor forcamera channel D represents a compensated version of the image capturedby camera channel D and is in the form of a data stream P_(D1), P_(D2),. . . P_(Dn).

The image plane integrator receives the data from each of the two ormore channel processors and combines such data into a single data set,e.g., P_(A1), P_(B1), P_(C1), P_(D1), P_(A2), P_(B 2), P_(C2), P_(D2),P_(A3), P_(B3), P_(C3), P_(D3), P_(An), P_(Bn), P_(Cn), P_(Dn). FIG.111F shows one embodiment of the image plane integrator.

In this embodiment, the image plane integrator includes a multiplexerand a multi-phase phase clock.

The multiplexer has a plurality of inputs in0, in1, in2, in3, each ofwhich is adapted to receive a stream (or sequence) of multi-bit digitalsignals. The data stream of multi-bit signals, P_(A1), P_(A2), . . .P_(An), from the channel processor for camera channel A is supplied toinput in0. The data stream P_(B1), P_(B2), . . . P_(Bn) from the channelprocessor for camera channel B is supplied to input in1. The data streamP_(C1), P_(C2), . . . P_(Cn) from the channel processor for camerachannel C is supplied to input in2. The data stream P_(D1), P_(D2), . .. P_(Dn) from the channel processor for camera channel D is supplied tothe input in3. The multiplexer has an output, out, that supplies amulti-bit output signal. Note that in some embodiments, the multiplexercomprises of a plurality of four input multiplexers each of which is onebit wide.

The multi-phase clock has an input enable that receives a signal. Themulti-phase clock has outputs, c0, c1, which are supplied to the inputss0, s1 of the multiplexer. In this embodiment, the multi-phase clock hasfour phases, shown in FIG. 111G.

The operation of the image plane integrator is as follows. Theintegrator has two states. One state is a wait state. The other state isa multiplexing state. Selection of the operating state is controlled bythe logic state of the enable signal supplied to the multi-phase clock.The multiplexing state has four phases, which correspond to the fourphases of the multi-phase clock. In phase 0, neither of the clocksignals c1, c0 are asserted causing the multiplexer to output one of themulti-bit signals from the A camera channel, e.g., P_(A1). In phase 1,clock signals c0, is asserted causing the multiplexer to output one ofthe multi-bit signals from the B camera channel, e.g., P_(B1). In phase2, clock signal c1, is asserted causing the multiplexer to output one ofthe multi-bit signals from the C camera channel, e.g., P_(C1). In phase0, both of the clock signals c1, c0 are asserted causing the multiplexerto output one of the multi-bit signals from the D camera channel, e.g.,P_(D1).

Thereafter, the clock returns to phase 0, causing the multiplexer tooutput another one of the multi-bit signals from the A camera channel,e.g., P_(A2). Thereafter, in phase 1, the multiplexer outputs anotherone of the multi-bit signals from the B camera channel, e.g., P_(B2). Inphase 2, the multiplexer outputs another one of the multi-bit signalsfrom the C camera channel, e.g., P_(C2). In phase 3, the multiplexeroutputs another one of the multi-bit signals from the D camera channel,e.g., P_(D2).

This operation is repeated until the multiplexer has output the lastmulti-bit signal from each of the camera channels, e.g., P_(An), P_(Bn),P_(Cn), and P_(Dn).

The output of the image plane integrator is supplied to the image planesalignment and stitching portion. The purpose of this portion is todetermine how the images should be aligned so that a target captured bydifferent camera channels is aligned at the same position within therespective images e.g., to make sure that a target captured by differentcamera channels appears at the same place within each of the camerachannel images).

For example, our eyes are good example of 2 channel image plane system.When we hold a pencil about 1 ft in front of our eyes, close our lefteye and use our right eye to see the pencil, we will see the pencil at aparticular location that is different than when we close our right eyeand use the left eye to see the pencil. This is because our brain onlyreceives 1 image at a time and couldn't correlate it with the otherimage from the other eye that was received at different time. When weopen both of our eyes and try to see the pencil again (hold it at thesame location as previous experiment), our brain will receive the 2images of the pencil at the same time. In this case, our brain willautomatically try to align the 2 images of the same pencil and we willperceive a single image of a pencil in front of us, except this onebecomes a stereo image.

In the case of the digital camera apparatus, the automatic image planesalignment and stitching portion determines how the 2, 3, 4, 5 or moreimage channels should be aligned.

FIGS. 111H-111J are explanatory views showing representations of imagesgenerated by three camera channels, e.g., camera channels 350A, 350B,350C, respectively, arranged in a triangular constellation in accordancewith one embodiment of the present invention and employed in oneembodiment of the automatic image planes alignment and stitchingportion.

Each image has a plurality of pixels arranged in a plurality of rows.More particularly, the image for a first camera channel, e.g., camerachannel 350A, has rows 1-n. The image for a second camera channel, e.g.,camera channel 350B, has rows 1-n. The image for a third camera channel,e.g., camera channel 350C, has rows 1-n. A reference line identifies ahorizontal reference point (e.g., midpoint) in the image for the firstcamera channel. A reference line identifies a horizontal reference point(e.g., midpoint) in the image for the second camera channel. A referenceline identifies a horizontal reference point (e.g., midpoint) in theimage for the third camera channel.

An object appears in each of the three images. In this embodiment, theobject appears at a different position in each image, for example as aresult of spatial offset between the camera channels. For example, theobject has two edges that intersect at an apex. In the image for thefirst camera channel, e.g., camera channel 350A, the apex appears in row2 and in line with the horizontal reference point. In the image for thesecond camera channel, e.g., camera channel 350B, the apex appears inrow 3 and to the left of the horizontal reference point. In the imagefor the third camera channel, e.g., camera channel 350C, the apexappears in row 3 and to the right of the horizontal reference point.

FIGS. 111K-111Q are explanatory views showing a representation of aprocess carried out by the automatic image alignment portion for thesystem with three camera channels, in accordance with one embodiment ofthe present invention. In this embodiment, the automatic image alignmentperforms vertical and horizontal alignment.

In that regard, vertical alignment may be performed first, although anyorder could be employed. The portion uses one of the images (e.g., theimage for the first camera channel, e.g., camera channel 350A) as areference image for comparison with the other images. The automaticimage alignment portion may initially compare row 1 of the referenceimage to row 1 of the other images and determines whether such rows ofsuch images defines a similar edge feature. In this example, none of theimages have an edge feature in the first row and thus there is not asimilar edge feature in each of such rows. As a result, the portionoutputs data corresponding to such rows (i.e., row 1 of each of thethree images) to the image scaling portion. In the next comparisonoperation, the automatic image alignment portion compares row 1 of thefirst image to row 2 of the other images. In this example, none of suchrows have an edge feature and thus there is not a similar edge featurein each of such rows. As a result, the portion outputs datacorresponding to such rows (i.e., row 1 of the first image and row 2 ofeach of the other images) to the image scaling portion. In the nextcomparison operation, the automatic image alignment portion compares row1 of the first image to row 3 of the other images. Although row 3 of theimages for the second and third channels each have an edge feature, row1 the image for the first channel does not have an edge.

The maximum number of comparison operations that use a particular row(of the reference image) may be selected based on the physical spacingbetween the camera channels. In this embodiment, for example, aparticular row of the reference image is used for three comparisonoperations at most. Thus, in subsequent comparison operations, theautomatic image alignment uses row 2 of the image for the first camerachannel, rather than row 1 of the image for the first camera channel. Inthe next comparison operation, the automatic image alignment portioncompares row 2 of the first image to row 2 of the other images. Althoughrow 2 of the image for the first camera channel has an edge, row 2 ofthe other camera channels does not have any edge. In the next comparisonoperation, the automatic image alignment portion compares row 2 of thefirst image to row 3 of the other images. In this example, each of suchrows has a similar edge feature. The automatic image alignment portionuses this as an indication of an overlapping image (or portion thereof).

Horizontal alignment is then performed. The portion determines themagnitude and the direction by which the image for the second channeland the image for the third channel should be shifted to align the edgefeature in such images with the edge feature in the image for the firstcamera channel and determines the width of the overlap of the images(e.g., the extent by which the images overlap in the horizontaldirection).

In the next comparison operation, the portion compares the next row ofthe reference image (e.g., row 3) to the next rows of the other images(e.g., row 4) and repeats the operations set forth above to determine aminimum width of the overlap of the images.

The images may be cropped in accordance with the vertical overlap andthe minimum horizontal overlap. The output of the automatic imagealignment portion is a cropped aligned image, which is supplied to theimage scaling portion. In some embodiments, the image scaling portionenlarges (e.g., upsamples) the cropped aligned image to generate animage that has the same size as that of the original images.

Some embodiments employ additional alignment methods, alone or incombination with any of the methods described herein. For example, insome embodiments, the above method is used in cases when the objects arerelatively far from the camera and other methods are used when objectsare relatively close to the camera.

FIG. 111AF shows a flowchart of operations that may be employed in thealignment portion, in accordance with another embodiment of the presentinvention. This embodiment of alignment may be used for example, forimages that include one or more close objects.

In this embodiment, edges are extracted at one of the planes.Neighborhood pixels (kernel) are defined for each edge pixel.Thereafter, the kernels of each edge pixel may be matched with pixels inthe other color planes, for example, by shifting the kernel towards thedirection where the other color plane is relatively located. One or moredeterminations may be made as to how well the kernels of each edge pixelare matched with pixels in the other color plane. In this regard, amatching cost function may be employed to quantify how well the kernelsof each edge pixel are matched with pixels in the other color plane. Inthe course of determining the best positions of each edge in the nextplane, the relative positions of each edge may be checked to confirmthat they preserve the same structure after shifting according to thebest matches.

After the final positions of the edges are set, the intervals betweenedges may be mapped, for example, using a linear mapping and/orshifting. Post processing may be performed on the shift amounts toconfirm that there are no outliers (no unexpected shifts relative to thesurrounding pixels).

The initial matching of the first two color planes may be employed as areference in regard to how much shift to expect at each pixel in theother color planes.

The above operations may be applied, for example, between the initialcolor plane and all the other color planes.

It should be understood that the automatic image alignment portion isnot limited to the embodiment above. For example, in some embodiments,fewer than three or more than three camera channels are aligned.Moreover, any other technique may be employed to align two or moreimages.

The alignment carried out by the automatic image alignment portion maybe predetermined, processor controlled and/or user controlled. In someembodiments, the automatic alignment portion has the ability to alignfewer than all of the camera channels (e.g., any two or more). In suchembodiments, one or more signals may be supplied to the automatic imagealignment portion to indicate the camera channels to align, and theautomatic image alignment portion may align the indicated camerachannels in response, at least in part, to such one or more signals. Theone or more signals, may be predetermined or adaptively determined,processor controlled and/or user controlled.

It should also be understood that automatic image alignment may not berequired in every embodiment.

The output of the image planes alignment and stitching is supplied tothe exposure control, the purpose of which is to help make sure that thecaptured images are not over exposed or under exposed. An over exposedimage is too bright. An under exposed image is too dark.

FIG. 111R shows one embodiment of the automatic exposure control. Insome embodiments, the auto exposure control generates a brightness valueindicative of the brightness of the image supplied thereto. The autoexposure control compares the generated brightness value against one ormore reference values, e.g., two values where the first value isindicative of a minimum desired brightness and the second value isindicative of a maximum desired brightness. The minimum and/or maximumbrightness may be predetermined, processor controlled and/or usercontrolled. In some embodiments, for example, the minimum desiredbrightness and maximum desired brightness values are supplied by theuser so that images provided by the digital camera apparatus will not betoo bright or too dark, in the opinion of the user.

If the brightness value is within the minimum desired brightness andmaximum desired brightness (i.e., greater than or equal to the minimumand less than or equal to the maximum), then the auto exposure controldoes not change the exposure time. If the brightness value is less thanthe minimum desired brightness value, the auto exposure control suppliescontrol signals that cause the exposure time to increase until thebrightness is greater than or equal to the minimum desired brightness.If the brightness value is greater than the maximum brightness value,then the auto exposure control supplies control signals that cause theexposure time to decrease until the brightness is less than or equal tothe maximum brightness value. After the brightness value is within theminimum and maximum brightness values (i.e., greater than or equal tothe minimum and less than or equal to the maximum), the auto exposurecontrol supplies a signal that enables a capture mode, wherein the useris able to press the capture button to initiate capture of an image andthe setting for the exposure time causes an exposure time that resultsin a brightness level (for the captured image) that is within the userpreferred range. In some embodiments, the digital camera apparatusprovides the user with the ability to manually adjust the exposure timedirectly, similar to adjusting an iris on a conventional film camera.

In some embodiments, the digital camera apparatus employs relativemovement between an optics portion (or one or more portions thereof) anda sensor array (or one or more portions thereof), to provide amechanical iris for use in auto exposure control and/or manual exposurecontrol. As stated above, such movement may be provided for exampleusing actuators, e.g., MEMS actuators and by applying appropriatecontrol signal(s) to one or more of the actuators to cause the one ormore actuators to move, expand and/or contract to thereby move theassociated optics portion.

As with each of the embodiments disclosed herein, the above embodimentsmay be employed alone or in combination with one or more otherembodiments disclosed herein, or portions thereof.

In addition, it should also be understood that the embodiments disclosedherein may also be used in combination with one or more other methodsand/or apparatus, now known or later developed.

As mentioned above, the inventions described and illustrated in the U.S.Provisional Application Ser. No. 60/695,946, entitled “Method andApparatus for use in Camera and Systems Employing Same”, filed Jul. 1,2005, may be employed in conjunction with the present inventions. Forthe sake of brevity, those discussions will not be repeated. It isexpressly noted that the entire contents of the aforementioned U.S.Provisional Application, including, for example, the features,attributes, alternatives, materials, techniques and/or advantages of allof the inventions/embodiments thereof, are incorporated by referenceherein.

The output of the exposure control is supplied to the Auto/Manual focuscontrol portion, which helps make the objects (e.g., the target(s) of animage) that are within the field of view appear in focus. Generally,objects in an image appear blurred if the image is over focus or underfocus. The image may have peak sharpness when the lens is in focuspoint. In some embodiments, the auto focus control portion detect theamount of blurriness of an image, e.g., while the digital cameraapparatus is in a preview mode, and provides control signals that causethe lens assembly to move back and forth, accordingly, until the autofocus control portion determines that the lens is at the focus point.Many of the digital still cameras available today utilize such type ofmechanism.

In some embodiments, the auto/manual focus portion is adapted to helpincrease the Depth of Focus of the digital camera apparatus. Depth ofFocus can be viewed as a measure of how much an object that is in focuswithin a field of view can be moved forward or backward before theobject becomes “out of focus”. Depth of Focus is based at least in parton the lens employed in the optical portion. Some embodiments employ oneor more optical filters in combination with a one or more algorithms toincrease the Depth of Focus. The optical filter or filters may beconventional optical filters for increasing Depth of Focus and may bedisposed superjacent (on or above) the top of the lens, although this isnot required. Any type of optical filter and positioning thereof may beemployed. Similarly, the algorithm or algorithms may be a conventionalwave front encoding algorithm, although this is not required. Any typeof algorithm or algorithms may be employed. In some embodiments, theauto focus mechanism increases the Depth of Focus by a factor of ten(e.g., the Depth of Focus provided with the auto focus mechanism is tentime as large as the Depth of Focus of the lens alone (without the autofocus mechanism), to make the system less sensitive or insensitive tothe position of objects within a field of view. In some embodiments, theauto focus mechanism increases the Depth of Focus by a factor of twentyor more (e.g., the Depth of Focus provided with the auto focus mechanismis twenty time as large as the Depth of Focus of the lens alone (withoutthe auto focus mechanism), to further decrease the sensitivity of theposition of the object within a field of view and/or to make the systeminsensitive to the position of objects within a field of view.

In some embodiments, the digital camera apparatus may provide the userwith the ability to manually adjust the focus.

In some embodiments, the digital camera apparatus employs relativemovement between an optics portion (or one or more portions thereof) anda sensor array (or one or more portions thereof), to help provide anauto focus and/or manual focus. As stated above, such movement may beprovided for example using actuators, e.g., MEMS actuators and byapplying appropriate control signal(s) to one or more of the actuatorsto cause the one or more actuators to move, expand and/or contract tothereby move the associated optics portion. (See, for example, U.S.Provisional Application Ser. No. 60/695,946, entitled “Method andApparatus for use in Camera and Systems Employing Same”, filed Jul. 1,2005, which is again incorporated by reference).

The auto/manual focus is not limited to the above embodiments. Indeed,any other type of auto/manual focus now known or later developed may beemployed.

In addition, as with each of the embodiments disclosed herein, the aboveembodiments may be employed alone or in combination with one or moreother embodiments disclosed herein, or portions thereof.

It should be understood that each of the embodiments disclosed hereinmay also be used in combination with one or more other methods and/orapparatus, now known or later developed.

It should also be understood that auto focus and manual focus are notrequired. Further, the focus portion may provide auto focus withoutregard to whether the ability to manual focus is provided. Similarly,the focus portion may provide manual focus without regard to whether theability to auto focus is provided.

The output of the auto focus control is supplied to the zoom controller.

FIG. 111S is schematic block diagram of one embodiment of the zoomcontroller, which may for example, help provide “optical zoom” and/or“digital zoom” capability. The optical zoom may be any type of opticalzoom now know or later developed. An example of conventional opticalzoom (which moves the one or more lens elements backward and forward) isdescribed herein above. Similarly, the digital zoom may be any type ofdigital zoom now know or later developed. Note that the determination ofthe desired zoom window may be predetermined, processor controlledand/or user controlled.

One drawback to digital zooming is a phenomenon referred to as aliasing.For example, when a television anchor on a news channels wears a stripedtie, the television image of the striped tie sometimes includes colorphenomena that do not appear on the actual tie. Aliasing of this type iscommon when a system does not have sufficient resolution to accuratelyrepresent one or more features of an object within the field of view. Inthe above example, the television camera does not have enough resolutionto accurately capture the striped pattern on the tie.

In some embodiments, the digital camera apparatus employs relativemovement between an optics portion (or one or more portions thereof) anda sensor array (or one or more portions thereof), to help increaseresolution, thereby helping to reduce and/or minimize aliasing thatmight otherwise occur as a result of digital zooming. As stated above,such relative movement may be provided for example using actuators,e.g., MEMS actuators and by applying appropriate control signal(s) toone or more of the actuators to cause the one or more actuators to move,expand and/or contract to thereby move the associated optics portion.

In some embodiments, for example, an image is captured and an opticsportion is thereafter moved in the x direction by a distance equal to ½of the width of a pixel. An image is captured with the optics in the newposition. The captured images may be combined to increase the effectiveresolution. In some embodiments, the optics portion is moved in the ydirection instead of the x direction. In some other embodiments, theoptics portion is moved in the x direction and the y direction and animage is captured at such position. In further embodiments, an image isalso captured at all four positions (i.e., no movement, moved in xdirection, moved in y direction, moved in x direction and y direction)and the images are then combined to further increase the resolution andfurther help reduce, minimize and or eliminate aliasing as a result ofzooming. For example, by doubling the resolution, it may be possible tozoom in by a factor of two without significantly increasing thealiasing.

In some embodiments, the relative movement is in the form of a ⅓ pixel×⅓pixel pitch shift in a 3×3 format. In some embodiments, it may bedesirable to employ a reduced optical fill factor. In some embodiments,one or more of the sensor arrays provides enough resolution to allow thedigital camera apparatus to perform digital zoom without excessivealiasing. For example, if an embodiment requires 640×480 pixels for eachevery image, with or without zoom, one or more of the sensor arrays maybe provided with 1280×1024 pixels. In such embodiment, such sensorportion(s) have enough pixels to provide the digital camera apparatuswith the resolution needed to zoom on ¼ of the image and yet stillprovide the required resolution of 640×480 pixels (e.g., ½×1280=640,½×1024=512).

FIGS. 111T-111V are explanatory views of a process carried out by a zoomportion of a digital camera apparatus in accordance with one suchembodiment of the present invention. In some embodiments, the subsystemmay use only ¼ of the pixels (e.g.,., ½×1280=640, ½×1024=512) when notin zoom mode, or may employ downsampling to reduce the number of pixels.In some other of such embodiments, the digital camera apparatus outputall of the pixels, e.g., 1280×1024, even when not in zoom mode. Thedetermination as to how many pixels to use and the number of pixels tooutput when not in zoom mode may be predetermined, processor controlledand/or user controlled.

The output of the zoom controller is supplied to the gamma correctionportion, which helps to map the values received from the camera channelsinto values that more closely match the dynamic range characteristics ofa display device (e.g., a liquid crystal display or cathode ray tubedevice). The values from the camera channels are based, at least inpart, on the dynamic range characteristics of the sensor, which oftendoes not match the dynamic range characteristics of the display device.The mapping provided by gamma correction portion helps to compensate forthe mismatch between the dynamic ranges.

FIG. 111W is a graphical representation showing an example of theoperation of the gamma correction portion.

FIG. 111X shows one embodiment of the gamma correction portion. In thisembodiment, the gamma correction portion employs a conventional transferfunction to provide gamma correction. The transfer function may be anytype of transfer function including a linear transfer function, anon-linear transfer function and/or combinations thereof. The transferfunction may have any suitable form including but not limited to one ormore equations, lookup tables and/or combinations thereof. The transferfunction may be predetermined, adaptively determined and/or combinationsthereof.

The output of the gamma correction portion is supplied to the colorcorrection portion, which helps to map the output of the camera into aform that matches the color preferences of a user.

In this embodiment, the color correction portion generates correctedcolor values using a correction matrix that contains a plurality ofreference values to implement color preferences as follows (Thecorrection matrix contains sets of parameters that are defined, forexample, by the user and/or the manufacturer of the digital camera):$\begin{pmatrix}{Rc} \\{Gc} \\{Bc}\end{pmatrix} = {\begin{pmatrix}{Rr} & {Gr} & {Br} \\{Rg} & {Gg} & {Bg} \\{Rb} & {Gb} & {Bb}\end{pmatrix} \times \begin{pmatrix}R \\G \\B\end{pmatrix}}$

such that:R corrected=(Rr×R un-corrected)+(Gr×G un-corrected)+(Br×B un-corrected),G corrected=(Rg×R un-corrected)+(Gg×G un-corrected)+(Bg×B un-corrected),andB corrected=(Rb×R un-corrected)+(Gb×G un-corrected)+(Bb×B un-corrected)

where

-   -   Rr is a value indicating the relationship between the output        values from the red camera channel and the amount of red light        desired from the display device in response thereto,    -   Gr is a value indicating the relationship between the output        values from the green camera channel and the amount of red light        desired from the display device in response thereto,    -   Br is a value indicating the relationship between the output        values from the blue camera channel and the amount of red light        desired from the display device in response thereto,    -   Rg is a value indicating the relationship between the output        values from the red camera channel and the amount of green light        desired from the display device in response thereto,    -   Gg is a value indicating the relationship between the output        values from the green camera channel and the amount of green        light desired from the display device in response thereto,    -   Bg is a value indicating the relationship between the output        values from the blue camera channel and the amount of green        light desired from the display device in response thereto,    -   Rb is a value indicating the relationship between the output        values from the red camera channel and the amount of blue light        desired from the display device in response thereto,    -   Gb is a value indicating the relationship between the output        values from the green camera channel and the amount of blue        light desired from the display device in response thereto, and    -   Bb is a value indicating the relationship between the output        values from the blue camera channel and the amount of blue light        desired from the display device in response thereto.

FIG. 111Y shows one embodiment of the color correction portion. In thisembodiment, the color correction portion includes a red color correctioncircuit, a green color correction circuit and a blue color correctioncircuit.

The red color correction circuit includes three multipliers. The firstmultiplier receives the red value (e.g., P_(An)) and the transfercharacteristic Rr and generates a first signal indicative of the productthereof. The second multiplier receives the green value (e.g., P_(Bn))and the transfer characteristic Gr and generates a second signalindicative of the product thereof. The third multiplier receives thegreen value (e.g., P_(Cn)) and the transfer characteristic Br andgenerates a third signal indicative of the product thereof. The first,second and third signals are supplied to an adder which produces a sumthat is indicative of a corrected red value (e.g., P_(An corrected)).

The green color correction circuit includes three multipliers. The firstmultiplier receives the red value (e.g., P_(An)) and the transfercharacteristic Rg and generates a first signal indicative of the productthereof. The second multiplier receives the green value (e.g., P_(Bn))and the transfer characteristic Gg and generates a second signalindicative of the product thereof. The third multiplier receives thegreen value (e.g., P_(Cn)) and the transfer characteristic Bg andgenerates a third signal indicative of the product thereof. The first,second and third signals are supplied to an adder which produces a sumindicative of a corrected green value (e.g., P_(Bn corrected)).

The blue color correction circuit includes three multipliers. The firstmultiplier receives the red value (e.g., P_(An)) and the transfercharacteristic Rb and generates a first signal indicative of the productthereof. The second multiplier receives the green value (e.g., P_(Bn))and the transfer characteristic Gb and generates a second signalindicative of the product thereof. The third multiplier receives thegreen value (e.g., P_(Cn)) and the transfer characteristic Bb andgenerates a third signal indicative of the product thereof. The first,second and third signals are supplied to an adder which produces a sumindicative of a corrected blue value (e.g., P_(Cn corrected)).

The output of the color corrector is supplied to the edgeenhancer/sharpener, the purpose of which is to help enhance featuresthat may appear in an image.

FIG. 111Z shows one embodiment of the edge enhancer/sharpener. In thisembodiment, the edge enhancer/sharpener comprises a high pass filterthat is applied to extract the details and edges and apply theextraction information back to the original image.

The output of the edge enhancer/sharpener is supplied to a random noisereduction portion, which reduces random noise in the image. Random noisereduction may include, for example, a linear or non-linear low passfilter with adaptive and edge preserving features. Such noise reductionmay look at the local neighborhood of the pixel in consideration. In thevicinity of edges, the low pass filtering may be carried out in thedirection of the edge so as to prevent blurring of such edge. Someembodiments may apply an adaptive scheme. For example, a low pass filter(linear and/or non linear) with a neighborhood of relatively large sizemay be employed for smooth regions. In the vicinity of edges, a low passfilter (linear and/or non-linear) and a neighborhood of smaller size maybe employed, for example, so as not to blur such edges.

Other random noise reduction may also be employed, if desired, alone orin combination with one or more embodiments disclosed herein. In someembodiments, random noise reduction is carried out in the channelprocessor, for example, after deviant pixel correction. Such noisereduction may be in lieu of, or in addition to, any random noisereduction that may be carried out in the image pipeline.

The output of the random noise reduction portion is supplied to thechroma noise reduction portion, the purpose of which is to reduce colornoise.

FIG. 111AA shows one embodiment of the chroma noise reduction portion.In this embodiment, the chroma noise reduction portion includes an RGBto YUV converter, first and second low pass filters and a YUV to RGBconverter. The output of the random noise reduction portion, which is asignal in the form of RGB values, is supplied to the RGB to YUVconverter, which generates a sequence of YUV values in response thereto,each YUV value being indicative of a respective one of the RGB values.

The Y values or components (which indicate the brightness of an image)are supplied to the YUV to RGB converter. The U and V values orcomponents (which indicate the color components of the image) aresupplied to the first and second low pass filters, respectively, whichreduce the color noise on the U and V components, respectively. Theoutput of the filters are supplied to the YUV to RGB converter, whichgenerates a sequence of RGB values in response thereto, each RGB valuebeing indicative of a respective one of the YUV values.

The output of the chroma noise reduction portion is supplied to theAuto/Manual white balance portion, the purpose of which is to help makesure that a white colored target appears as a white colored target,rather than reddish, greenish, or bluish.

FIG. 111AB is an explanatory view showing a representation of a processcarried out by the white balance portion in one embodiment. Moreparticularly, FIG. 111AB depicts a rectangular coordinate plane havingan R/G axis and a B/G axis. The rectangular coordinate plane has threeregions, i.e., a redish region, a white region and a bluish region. Afirst reference line defines a color temperature that separates theredish region from the white region. A second reference line defines acolor temperature that separates the white region from the bluishregion. The first reference line is disposed, for example at colortemperature of 4700 Kelvin. The second reference line is disposed, forexample at color temperature of 7000 Kelvin.

In this embodiment, the automatic white balance portion determines thepositions, in the rectangular coordinate plane defined by the R/G axisand the B/G axis, of a plurality of pixels that define the originalimage. The positions of the plurality of pixels are treated asrepresenting a cluster of points in the rectangular coordinate plane.The automatic white balance portion determines a center of the clusterof points and changes that could be applied to the R, G, B, pixel valuesof the original image to effectively translate the center of the clusterinto the white image region of the coordinate plane, e.g., to a colortemperature of 6500 Kelvin. The output of the automatic white balanceportion is an output image where a pixel value in the output image isbased on the corresponding pixel value of the original image and thechanges to the R, G, B pixel values that had been determined could beused to translate the center of the cluster for the original image intothe white region, such that the center of a cluster for the output imageis disposed in the white image region of the coordinate plane, e.g., acolor temperature of 6500 Kelvin.

The desired color temperature may be predetermined, processor controlledand/or user controlled. In some embodiments, for example, a referencevalue indicative of a desired color temperature is supplied by the userso that images provided by the digital camera apparatus will have colortemperature characteristics desired by the user. In such embodiments,manual white balance may be performed by determining the changes thatcould be applied to translate the center of the cluster for the originalimage to a color temperature corresponding to a reference value providedby the user.

The white balance strategy may use, for example, one or moreconventional color enhancement algorithms, now know or later developed.

It should be understood that the white balance portion is not limited tothe techniques set forth above. Indeed, the white balance portion mayemploy any white balance technique now known or later developed. Itshould also be understood that color white balance is not required. Theoutput of the white balance portion is supplied to the Auto/Manual colorenhancement portion.

FIG. 111AC is a block diagram of one embodiment of the color enhancementportion, in accordance with one embodiment. In this embodiment, thecolor enhancement portion adjusts the brightness, contrast and/orsaturation to enhance the color appearance in accordance with one ormore enhancement strategies. This process is similar in some respects toadjusting color settings of a TV or computer monitor. Some embodimentsmay also adjust the hue. The enhancement strategy may use, for example,one or more conventional color enhancement algorithms, now know or laterdeveloped.

Referring to FIG. 111AC, data indicative of the image is supplied to thebrightness enhancement portion, which further receives an adjustmentvalue and generates output data indicative of an image adjusted forbrightness in accordance therewith. In this embodiment, each pixel valuein the output image is equal to the sum of an adjustment value and acorresponding pixel in the input image. The adjustment value may bepredetermined, processor controlled and/or user controlled. In someembodiments, for example, the adjustment value is supplied by the userso that images provided by the digital camera apparatus will have thecharacteristics desired by the user. In some embodiments, an adjustmentvalue having a positive magnitude makes the output image appear brighterthan the input image. An adjustment value having a negative magnitudemay make the output image appear darker than the input image.

The output of the brightness enhancement portion is supplied to thecontrast enhancement portion, which further receives an adjustment valueand generates an output image adjusted for contrast in accordancetherewith. In this embodiment, contrast adjustment can be viewed as“stretching” the distance between dark (e.g., indicated by a pixel valuehaving a small magnitude) and light (e.g., indicated by a pixel valuehaving a large magnitude). An adjustment value having a positivemagnitude makes dark areas in the input image appear darker in theoutput image and makes light areas in the input image appear lighter inthe output image. An adjustment value having a negative magnitude mayhave the opposite effect. One or more conventional algorithms, forexample, now know or later developed may be employed. The adjustmentvalue may be predetermined, processor controlled and/or user controlled.In some embodiments, for example, the adjustment value is supplied bythe user so that images provided by the digital camera apparatus willhave the characteristics desired by the user.

The output of the contrast enhancement portion is supplied to thesaturation enhancement portion, which further receives an adjustmentvalue and generates an output image adjusted for saturation inaccordance therewith. In this embodiment, saturation adjustment can beviewed as “stretching” the distance between R, G, B, components of apixel (which is similar in some respects to contrast adjustment). Anadjustment value having a positive magnitude makes dark areas in theinput image appear darker in the output image and makes light areas inthe input image appear lighter in the output image. An adjustment valuehaving a negative magnitude may have the opposite effect. One or moreconventional techniques, for example, now know or later developed may beemployed. The technique may employ a color correction matrix, forexample, similar to that employed by the color correction portiondescribed hereinabove. The adjustment value may be predetermined,processor controlled and/or user controlled. In some embodiments, forexample, the adjustment value is supplied by the user so that imagesprovided by the digital camera apparatus will have the characteristicsdesired by the user.

It should be understood that the color enhancement portion is notlimited to the enhancement techniques set forth above. Indeed, the colorenhancement portion may employ any enhancement technique now known orlater developed. It should also be understood that color enhancement isnot required.

The output of the Auto/Manual color enhancement portion is supplied tothe image scaling portion, the purpose of which is to reduce or enlargethe image, for example, by removing or adding pixels to adjust the sizeof an image.

The image scaling portion receives data, indicative of an imaged to bescaled (e.g., enlarged or reduced). The magnitude of the scaling may bepredetermined or preset, processor controlled or manually controlled. Insome embodiments, a signal indicative of the magnitude of the scaling,if any, is received. If the signal indicative of the desired scalingmagnitude indicates that the image is to be enlarged, then the scalingportion performs upscaling. If the signal indicative of the desiredscaling magnitude indicates that the image is to be reduced, then thescaling portion performs downscaling.

FIG. 111AD-111AE are a schematic block diagram and an explanatory view,showing a representation of upscaling, respectively, in accordance withone embodiment. More particularly, FIG. 111AE depicts a portion of animage to be enlarged and a portion of the image to be formed therefrom.In this example, the portion of the image to be enlarged includes ninepixels, indicated for purposes of explanation as P₁₁-P₃₃, shown arrangedin an array having three rows and three columns. The portion of theimage to be formed therefrom includes twenty five pixels, indicated forpurposes of explanation as A-Y, shown arranged in an array having fiverows and five columns. (Note that the portion of the image to be formedcould alternatively be represented as P₁₁-P₅₅.)

In this embodiment, the image scaling portion employs an upscalingstrategy in which the pixel values at the intersection of an oddnumbered column and an odd numbered row, i.e., A, C, E, K, M, O, U, Wand Y, are taken from the pixel values in the image to be enlarged. Forexample,A=P₁₁C=P₂₁E=P₃₁K=P₁₂M=P₂₂O=P₃₂U=P₃W=P₂₃Y=P₃₃

The other pixel values, i.e., pixel values disposed in either an evennumbered column or an even numbered row, i.e., B, D, F, G, H, I, J, L,N, P, Q, R, S, T, V and X, are generated by interpolation. Each pixelvalue is generated based on two or more adjacent pixel values, forexample,

B=(A+C)/2D=(C+E)/2F=(A+K)/2H=(C+M)/2J=(E+O)/2L=(K+M)/2N=(M+O)/2P=(K+U)/2R=(M+W)/2T=(O+Y)/2V=(U+W)/2X=(W+Y)/2G=(B+L)/2I=(D+N)/2Q=(L+V)/2S=(N+X)/2

In some embodiments, upscaling increases the number of pixels from640×480 pixels to 1280×1024 pixels, however, any magnitude of upscalingmay be employed. In some embodiments, the digital camera apparatusprovides the user with the ability to determine whether upscaling is tobe performed and if so, the magnitude of the upscaling.

In some embodiments, the scaling portion employ one or more of thetechniques described herein for the zoom controller, with or withoutcropping.

It should be understood that the scaling portion is not limited to theupscaling strategy set forth above. Indeed, the scaling portion mayemploy any upscaling technique now known or later developed. It shouldalso be understood that upscaling is not required.

The scaling portion may have the ability to downscale, without regard towhether scaling portion has the ability to upscale. In some embodiments,downscaling decreases the number of pixels from 1280×1024 pixels to640×480 pixels, however, any magnitude of downscaling may be employed.In some embodiments, the digital camera apparatus provides the user withthe ability to determine whether downscaling is to be performed and ifso, the magnitude of the downscaling.

It should be understood that any downscaling technique now known orlater developed may be employed. It should also be understood thatdownscaling is not required.

The output of the image scaling portion is supplied to the color spaceconversion portion, the purpose of which is to convert color format fromRGB to YCrCB or YUV for compression. In this embodiment, the conversionis accomplished using the following equations:Y=(0.257*R)+(0.504*G)+(0.098*B)+16Cr=V=(0.439*R)−(0.368*G)−(0.071*B)+128Cb=U=−(0.148*R)−(0.291*G)+(0.439*B)+128

The output of the color space conversion portion is supplied to theimage compression portion of the post processor. The purpose of theimage compression portion is to reduce the size of image file. This maybe accomplished, or example, using an off the shelf JPEG, MPEG and/orWMV compression algorithm available from Joint Photographic ExpertGroup, Motion Expert Group and Microsoft Corporation.

The output of the image compression portion is supplied to the imagetransmission formatter, the purpose of which is to format the image datastream to comply with YUV422, RGB565, etc format both in bi-directionalparallel or serial 8-16 bit interface.

FIG. 112 shows another embodiment of the channel processor. In thisembodiment, the double sampler receives the output of the analog todigital converter instead of the output of the sensor array

FIGS. 113 and 114A show another embodiment of the channel processor andimage pipeline respectively. In this embodiment, the deviant pixelcorrector is disposed in the image pipeline rather than the channelprocessor. In this embodiment, the deviant pixel corrector receives theoutput of the image plane alignment and stitching rather than the outputof the black level clamp.

FIG. 114B is a block diagram of an image pipeline in accordance withanother embodiment of the present invention;

FIG. 114C is a schematic block diagram of a chroma noise reductionportion that may be employed, for example, in the image pipeline of FIG.114B. In this embodiment, the U and V values or components (whichindicate the color components of the image) are supplied to the firstand second low pass filters, respectively, which reduce the color noiseon the U and V components, respectively.

It should be understood that the channel processor, the image pipelineand/or the post processor may have any configuration. For example, insome other embodiments, the image pipeline employs fewer than all of theportions shown in FIGS. 110C, 110E and/or FIG. 114A, with or withoutother portions, now known or later developed, and in any suitable order.

Parallax

If the digital camera apparatus has more than one camera channel, thecamera channels will necessarily be spatially offset from one another(albeit, potentially by a small distance). This spatial offset canintroduce a parallax between the camera channels, e.g., an apparentchange in position of an object as a result of changing the positionfrom which the object is viewed.

FIGS. 115A-115E show an example of parallax in a digital cameraapparatus. More particularly, FIG. 115A shows an object (i.e., alightning bolt) and a digital camera apparatus having two camerachannels spatially offset by a distance. The first camera channel has asensor and a first field of view centered about a first axis. The secondcamera channel has a sensor and a second field of view that is centeredabout a second axis and spatially offset from the first field of view.The offset between the fields of view causes the position of the objectwithin the first field of view to differ from the position of the objectwithin the second field of view.

FIG. 115B is a representation of an image of the object, as viewed bythe first camera channel, striking the sensor in the first camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115C is a representation of an image of the object, as viewed bythe second camera channel, striking the sensor in the second camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115D shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel. In thisembodiment, the parallax is in the x direction.

FIG. 115E shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel if suchparallax is eliminated.

FIGS. 115F-115H show an example of parallax in the y direction. FIG.115I shows the image viewed by the first camera channel superimposedwith the image viewed by the second camera channel if such parallax iseliminated.

FIGS. 115J-115L show an example of parallax having an x component and ay component. FIG. 115M shows the image viewed by the first camerachannel superimposed with the image viewed by the second camera channelif such parallax is eliminated.

FIG. 115N shows an object (i.e., a lightning bolt) and a digital cameraapparatus having two camera channels spatially offset by a distance. Thefirst camera channel has a sensor and a first field of view centeredabout a first axis. The second camera channel has a sensor and a secondfield of view that is centered about a second axis and spatially offsetfrom the first field of view. The offset between the fields of viewcauses the position of the object within the first field of view todiffer from the position of the object within the second field of view.

FIG. 115O is a representation of an image of the object, as viewed bythe first camera channel, striking the sensor in the first camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115P is a representation of an image of the object, as viewed bythe second camera channel, striking the sensor in the second camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115Q shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel. In thisembodiment, the parallax is in the x direction.

FIG. 115R shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel if suchparallax is eliminated.

FIG. 115S shows an object (i.e., a lightning bolt) and a digital cameraapparatus having two camera channels spatially offset by a distance. Thefirst camera channel has a sensor and a first field of view centeredabout a first axis. The second camera channel has a sensor and a secondfield of view that is centered about a second axis and spatially offsetfrom the first field of view. The offset between the fields of viewcauses the position of the object within the first field of view todiffer from the position of the object within the second field of view.

FIG. 115R is a representation of an image of the object, as viewed bythe first camera channel, striking the sensor in the first camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115P is a representation of an image of the object, as viewed bythe second camera channel, striking the sensor in the second camerachannel. The sensor has a plurality of sensor elements shownschematically as circles.

FIG. 115Q shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel. In thisembodiment, the parallax is in the x direction.

FIG. 115R shows the image viewed by the first camera channelsuperimposed with the image viewed by the second camera channel if suchparallax is eliminated.

Range Finding

In some embodiments, it is desirable to be able to generate an estimateof the distance to an object within the field of view. This capabilityis sometimes referred to as “range finding”.

One method for determining an estimate of a distance to an object is toemploy parallax.

FIG. 116 shows a flowchart of operations that may be employed ingenerating an estimate of a distance to an object, or portion thereof,according to another embodiment of the present invention.

The system receives a signal indicative of a desired amount of parallaxand/or one or movements

The system identifies one or more movements to provide or help providethe desired amount of parallax.

The system initiates one, some or all of the one or more movements.

An image is captured from each camera channel to be used in generatingthe estimate of the distance to the object (or portion thereof). Forexample, if two camera channels are to be used in generating theestimate, then an image is captured from the first camera channel and animage is captured from the second camera channel.

In some embodiments, the system receives one or more signals indicativeof the position of the object in the images or determines the positionof the object within each image. For example, if two camera channels areto be used in generating the estimate of the distance to the object, thesystem may receive one or more signals indicative of the position of theobject in the image from the first camera channel and the position ofthe object in the image from the second camera channel. In some otherembodiments, the system determines the position of the object withineach image, e.g., the position of the object within the image for thefirst channel and the position of the object within the image for thesecond channel.

The system generates a signal indicative of the difference between thepositions in the images. For example, if two camera channels are beingused, the system generates a signal indicative of the difference betweenthe position of the object in the image for the first camera channel andthe position of the object in the image for the second camera channel.

The system generates an estimate of the distance to the object (orportion thereof) based at least in part on (1) the signal indicative ofthe difference between the position of the object in the image for thefirst camera channel and the position of the object in the image for thesecond camera channel (2) the signal indicative of the relativepositioning of the first camera channel and the second camera channeland (3) data indicative of a correlation between (a) the differencebetween the position of the object in the image for the first camerachannel and the position of the object in the image for second camerachannel, (b) the relative positioning of the first camera channel andthe second camera channel and (c) the distance to an object.

FIG. 117 is a block diagram showing a portion of one embodiment therange finder. In this embodiment, the range finder includes adifferencer and an estimator. The differencer has one or more inputsthat receive one or more signals indicative of the position of theobject in a first image and the position of the object in a secondimage. The differencer further includes one or more outputs that supplya difference signal, Difference. The difference signal, Difference, isindicative of the difference between the position of the object in thefirst image and the position of the object in the second image.

The difference signal is supplied to the estimator, which also receivesa signal indicative of the relative positioning between the camerachannel that provided the first image and the camera channel thatprovided the second image. In response, the estimator provides an outputsignal, estimate, indicate of an estimate of the distance to the object(or portion thereof).

In order to accomplish this, the estimator includes data indicative ofthe relationship between (a) the difference between the position of theobject in the first image and the position of the object in the secondimage, (b) the relative positioning of the camera channel generating thefirst image and the camera channel generating the second image and (c)the distance to an object.

This data may be in any form, including for example, but not limited to,a mapping of a relationship between inputs (e.g., (a) the differencebetween the position of the object in the first image and the positionof the object in the second image and (b) the relative positioning ofthe camera channel generating the first image and the camera channelgenerating the second image) and the output (e.g., an estimate of thedistance to the object).

The mapping may have any of various forms known to those skilled in theart, including but not limited to a formula and/or a look-up table. Themapping may be implemented in hardware, software, firmware or anycombination thereof.

The mapping is preferably generated “off-line” by placing an object at aknown distance from the digital camera apparatus, capturing two or moreimages with two or more camera channels having a known relativepositioning and determining the difference between the position of theobject in the image from the first camera channel and the position ofthe object in the image from the second camera channel.

This above process may be repeated so as to cover different combinationsof known distance to the object and relative positioning of the camerachannels. It may be advantageous to cover an entire range of interest(e.g. known distances and relative positioning), however, as explainedbelow, it is generally not be necessary to cover every conceivablecombination. Each combination of known distance to object, relativepositioning of camera channels and difference between the position ofthe object in the image from the first camera channel and the positionof the object in the image from the second camera channel represents onedata point in the overall input output relation.

The data points may be used to create a look-up table that provides, foreach of a plurality of combinations of input magnitudes, an associatedoutput. Or, instead of a look-up table, the data points may be input toa statistical package to produce a formula for calculating the outputbased on the inputs. The formula can typically provide an appropriateoutput for any input combination in the sensor input range of interest,including combinations for which data points were not generated.

A look-up table embodiment may employ interpolation to determine anappropriate output for any input combination not in the look-up table.

The differencer may be any type of differencer that is adapted toprovide one or more difference signals indicative of the differencebetween the position of the object in the first image and the positionof the object in the second image. In this embodiment, for example, thedifferencer comprises an absolute value subtractor that generates adifference signal equal to the absolute value of the difference betweenthe position of the object in the first image and the position of theobject in the second image. In some other embodiments, the differencermay be a ratiometric type of differencer that generates a ratiometricdifference signal indicative of the difference between the position ofthe object in the first image and the position of the object in thesecond image.

The signal indicative of the relative position of the camera channelsmay have any form. For example, the signal may be in the form of asingle signal that is directly indicative of the difference in positionbetween the camera channels. The signal may also be in the form of aplurality of signals, for example, two or more signals each of whichindicates the position of a respective one of the camera channels suchthat the plurality of signals are indirectly indicative of the relativeposition of the camera channels.

Although the portion of range finder is shown having a differencerpreceding the estimator, the range finder is not limited to such. Forexample, a differencer may be embodied within the estimator and/or adifference signal may be provided or generated in some other way. Insome embodiments, the estimator may be responsive to absolute magnitudesrather than difference signals.

Furthermore, while the disclosed embodiment includes three inputs andone output, the range finder is not limited to such. The range findermay be employed with any number of inputs and outputs.

Range finding may also be carried out using only one camera channel. Forexample, one of the camera channels may be provided with a first view ofan object and an image may be captured. Thereafter, one or moremovements may be applied to one or more portions of the camera channelso as to provide the camera channel with a second view of the object(the second view being different that the first view). Such movementsmay be provided by the positioning system. A second image may becaptured with the second view of the object. The first and second imagesmay thereafter be processed by the range finder using the operations setforth above to generate an estimate of a distance to the object (orportion thereof).

FIG. 118 is a block diagram representation of a locator portion of therange finder.

3D Imaging

Referring to FIGS. 119A-119D, in some embodiments, it is desired to beable to produce images for use in providing one or more 3D effects,sometimes referred to herein as “3D imaging”.

One type of 3D imaging is referred to as stereovision. Stereovision isbased, at least in part, on the ability to provide two views of anobject, e.g., one to be provided to the right eye, one to be providedthe left eye. In some embodiment, the views are combined into a singlestereo image. In one embodiment, for example, the view for the right eyemay be blue and the view for the left eye may be red, in which case, aperson wearing appropriate eyewear (e.g., blue eyepiece in front of lefteye, red eyepiece in front of right eye) will see the appropriate viewin the appropriate eye (i.e., right view in the right eye and the leftview in the left eye). In another embodiment, the view for the right eyemay be polarized in a first direction(s) and the view for the left eyemay be polarized in a second direction(s) different than the first, inwhich case, a person wearing appropriate eyewear (e.g., eyepiecepolarized in first direction(s) in front of left eye, eyepiece polarizedin second direction(s) in front of left eye) will see the appropriateview in the appropriate eye (i.e., right view in the right eye and theleft view in the left eye).

Referring to FIG. 120, another type of 3D imaging is referred to as 3Dgraphics, which is based, at least in part, on the ability to provide animage with an appearance of depth.

It is desirable to employ parallax when producing images for use inproviding 3D effects.

Increasing the amount of parallax may help improve one or morecharacteristics of the 3D imaging.

FIG. 121A-121B shows a flowchart of operations that may be employed inproviding 3D imaging, according to another embodiment of the presentinvention.

The system receives a signal indicative of a desired amount of parallaxand/or one or movements.

The system identifies one or more movements to provide or help providethe desired amount of parallax.

The system initiates one, some or all of the one or more movementsidentified.

The system generates one or more images with the desired 3D effect.

An image is captured from each camera channel to be used in the 3Dimaging. For example, if two camera channels are to be used in the 3Dimaging, then an image is captured from the first camera channel and animage is captured from the second camera channel.

The system determines whether stereovision is desired or whether 3Dgraphics is desired. If stereovision is desired, the image captured fromthe first camera channel and the image captured from the second camerachannel are each supplied to a formatter, which generates two images,one suitable to be provided to one eye and one suitable to be providedto the other eye. For example, in one embodiment, for example, the viewfor the right eye may be blue and the view for the left eye may be red,in which case, a person wearing appropriate eyewear will see theappropriate view in the appropriate eye (i.e., right view in the righteye and the left view in the left eye). In another embodiment, the viewfor the right eye may be polarized in a first direction(s) and the viewfor the left eye may be polarized in a second direction(s) differentthan the first, in which case, a person wearing appropriate eyewear willsee the appropriate view in the appropriate eye (i.e., right view in theright eye and the left view in the left eye).

The two images may be combined into a single stereo image.

If 3D graphics is desired instead of stereovision, the systemcharacterizes the images using one or more characterization criteria. Inone embodiment, for example, the characterization criteria includesidentifying one or more features (e.g., edges) in the images and anestimate of the distance to one or more portions of such features. Arange finder as set forth above may be used to generate estimates ofdistances to features or portions thereof. The system generates a 3Dgraphical image having the appearance of depth, at least in part, based,at least in part, on (1) the characterization data and (2) 3D renderingcriteria.

The characterization criteria and the 3D graphical criteria may bepredetermined, adaptively determined, and or combinations thereof.

It should be understood that 3D imaging may also be carried out usingonly one camera channel. For example, one of the camera channels may beprovided with a first view of an object and an image may be captured.Thereafter, one or more movements may be applied to one or more portionsof the camera channel so as to provide the camera channel with a secondview of the object (the second view being different that the firstview). Such movements may be provided by the positioning system. Asecond image may be captured with the second view of the object. Thefirst and second images may thereafter be processed by the 3D imagerusing the operations set forth above to generate an estimate of adistance to the object (or portion thereof).

FIG. 123 is a block diagram representation of one embodiment forgenerating an image with a 3D effect.

FIG. 124 is a block diagram representation of one embodiment forgenerating an image with 3D graphics.

Image Discrimination

FIG. 125 shows a flowchart of operations that may be employed inproviding image discrimination, according to another embodiment of thepresent invention.

FIGS. 126A-126B shows a flowchart of operations that may be employed inproviding image discrimination, according to another embodiment of thepresent invention.

Certain Other Applications

In some embodiments, the number of image sensors, size and/or type areselected based on application requirements. Described are three examplesand how it can impact camera elements, and if desired,feature/operational optimization. It should be understood that any ofthe above embodiments, or portions thereof, may be employed inimplementing any of the following examples.

#1): Simultaneous Imaging Hyper-Spectral Digital Camera:

Hyper-spectral imagers take data in as many as one hundred distinctcolor bands. This can be accomplished by electronically tunable ormechanically selected narrow band filters. One problem with this tunableor selectable filter approach is that the color bands in the image areselected time sequentially. It takes many frames of data to collect thefull hyper-spectral image (called a datacube) with 3-D pixelidentifiers: x, y and color. In many system applications, acquiring theentire hyper-spectral datacube simultaneously in one frame of data isdesired.

The disclosed multiple optical/imager approach can be used to acquireall color bands simultaneously using a separate color narrow band-passfilter in each sensor optical path. An example would be 64 individualsensors (custom optical assemblies, optional MEMs mechanical dithermechanisms and optimized color or multiple color imager sensors)arranged in, for example, a 8×8 or 1×64 or other sensor arrangements.This would give a hyper-spectral capability of sixty four unique colorbands. Each sensor would have some suitable number of pixels in theimage sensor to cover a desired field-of-view (for example: 256×256pixels on the imager array assembled with a 3 μm pixel pitch).

Each image sensor can have a different pixel pitch and/or array sizewith the imager integrated circuit (IC) optimized for the incident coloror color bands. If the frame rate of each sensor is 60 frames persecond, data in 64 unique color bands would be acquired in one frametime (16.67 msec). This capability or similar capability is desirablefor many hyper-spectral imaging applications.

#2): Object (Threat) Detection and Identification Multi-ColorActive/Passive Digital Camera:

Some camera systems need to acquire data from a wide field of view(WFOV) to detect an object of interest, then fast frame around thatobject in a narrow field of view (NFOV) with multiple color imagingcapability and higher spatial resolution capability to identify thatobject.

The WFOV sensor, for example, could have a 128×128 array size with20-micron pitch pixels to determine an object of interest within one ormore pixels in the WFOV. The optics and pixels in the 128×128 arraycould be broadband visible capable for high sensitivity.

The entire digital camera (with the WFOV image sensor and multiple NFOVimage sensors) can be pointed by a gimbal mechanism. The data from theWFOV image sensor can adjust the gimbal pointing direction such that thedetected object is in the center of all FOVs.

The NFOV image sensors with higher resolution than the WFOV imagersensor can image and identify the object. The WFOV image sensor cancontinue to image a WFOV for detection of other objects of interest.

There can be multiple NFOV image sensors. The NFOV can be selected bypixel size, number of pixels (in x and y) and the focal length of theoptics. As an example, the camera could contain six NFOV image sensors.The NFOV could image a region 1/100 that of the WFOV image sensor. Ifthe focal lengths of the WFOV and NFOV optics are the same, a 128×128image array with a 2.0 μm pixel pitch, for example, would provide thedesired NFOV.

The six NFOV image sensors can all be different. One example is imagesensors optimized for ultraviolet (UV), blue, green, red, broadbandvisible, 880 nm continuous laser illumination and 880 nm pulsed laserillumination. These six sensors can be optimizes for pixel size andarray size to match the NFOV. In general, the pixel pitch will increasefor longer wavelengths to match the optical blur circle. The pulsed 880nm laser array can have special circuitry within each pixel to measurethe amplitude and arrival time of the reflected laser pulse off theobject; this capability called LADAR, provides distance to object,reflected signal amplitude and in some cases 3D information of theobject's shape.

The WFOV imager sensor and the other six NFOV images sensors can beprocessed on a single integrated circuit. The location of the imagesensors on the integrated circuit can be chosen to minimize integratedcircuit area or for other considerations. Each sensor is optimized forits intended operation. The optical stack above each sensor provides thedesired color transmission and other desired optical features. Theoptical stacks or portions there of can be mechanically dithered by aMEMs mechanical mechanism if desired to achieve higher spatialresolution or provide other capability (such as image stabilization orimage focus).

The NFOV image sensors can go into a widowing readout with the reducedFOV (perhaps 32×32 pixels at 8× faster frame rate). The data from theNFOV sensors can be used to point the gimbal to keep the object ofinterest in the center of the reduced FOV.

#3: Large Dynamic Range Color Digital Camera

Digital cameras may have a maximum photo-signal storage capacity thatlimits the dynamic range of the particular system. The photo-signalcharge is stored on a capacitor within the pixel area. The chargehandling capacity is limited by the maximum voltage swing in theintegrated circuitry and the storage capacitance within the pixel. Theamount of integrated photo-charge is directly related to the time theimage sensor collects and integrates signal from the scene. This isknown as integration time. A long integration time is desired for weaksignals since more photo-charge is integrated within the pixel and thesignal-to-noise of the digital camera is improved.

Once a maximum charge capacity is reached, the sensor can no longer tellhow much brighter the image was. This creates an imaging dilemma bysetting a single integration time for the entire field of view. Thedigital camera's integration time can be set to image low light levelsand saturate bright signals or image high light levels and not detectlow light levels (since the integrated photo-charge from low lightlevels is below the signal-to-noise of the sensor).

The use of multiple optics and image sensors on a single IC, all lookingat the same field of view simultaneously, and each having differentintegration times, solves this dynamic range problem. The digital cameracould have, for example a 3×3 assembly of image sensors, perhaps threeof each color (R, G, and B) and the integration time in each color canbe varied, for example each color can have three distinct values(perhaps 0.1, 1 and 10 msec). The data from each color of cameras can bedigitally combined to provide a much greater dynamic range within oneframe of digital camera data. While it would be difficult to displaythis wide dynamic range of imagery without compression, the raw digitalcamera data could be used by digital signal processing of the scene. Thedigital data can also be stored and displayed to exhibit low light orbright light characteristics as desired.

The optical stack can also contain other optical features that aredesired for digital camera functionality and performance. This can bethings such as electronically tunable filters, polarizers, wavefrontcoding, spatial filters (masks), and other features not yet anticipated.Some of the new features (in addition to the lenses) can be electricallyoperated (such as a tunable filter) or be moved mechanically with MEMsmechanisms.

The manufacture of the optical stacks and image sensors may be done on asingle wafer, fabricated on separate wafers (perhaps two wafers: one forthe IC, and one for optics) and bonded together at the wafer level. Itis also possible to use pick and place methods and apparatus to attachthe optical assemblies to the wafer IC, or the image sensor die andother assemblies (optical stack) can be assembled individually

In embodiments that employ MEMS, manufacture of the optical stacks, MEMsand image sensors may be done on a single wafer, fabricated on separatewafers (perhaps up to three wafers: one for the IC, one for MEMs and onefor optics) and bonded together at the wafer level. It is also possibleto use pick and place methods and apparatus to attach the opticalassemblies and MEMs to the wafer IC, or the image sensor die and otherassemblies (MEMs and optical stack) can be assembled individually.

It should also be understood that although the digital camera apparatus210 is shown employed in a digital camera 200, the present invention isnot limited to such. Indeed, a digital camera apparatus and/or any ofthe methods and/or apparatus that may be employed therein may be used byitself or in any type of device, including for example, but not limitedto, still and video cameras, cell phones, other personal communicationsdevices, surveillance equipment, automotive applications, computers,manufacturing and inspection devices, toys, and/or a wide range of otherand continuously expanding applications. Moreover, other devices thatmay employ a digital camera apparatus and/or any of the methods and/orapparatus employed may or may not include the housing, circuit board,peripheral user interface, power supply, electronic image storage mediaand aperture depicted in FIG. 2 (for example, the circuit board may notbe unique to the camera function but rather the DCS be an add-on to anexisting circuit board, such as in a cell phone) and may or may notemploy methods and/or apparatus not shown in FIG. 2.

A digital camera may be a stand-alone product or may be imbedded inother appliances, such as cell phones, computers or the myriad of otherimaging platforms now available or may be created in the future, such asthose that become feasible as a result of this invention.

One or more embodiments of one or more aspects of the present inventionmay have one or more of the advantages below. A device according to thepresent invention can have multiple separate arrays on a single imagesensor, each with its own lens. The simple geometry of a smaller,multiple arrays allows for a smaller lens (diameter, thickness and focallength), which allows for reduced stack height in the digital camera.

Each array can advantageously be focused on one band of visiblespectrum. Among other things, each lens may be tuned for passage of thatone specific band of wavelength. Since each lens would therefore notneed to pass the entire light spectrum, the number of elements will bereduced, likely to one or two.

Further, due to the focused bandwidth for each lens, each of the lensesmay be dyed during the manufacturing process for its respectivebandwidth (e.g., red for the array targeting the red band of visiblespectrum). Alternatively, a single color filter may be applied acrosseach lens. This process eliminates the traditional color filters (thesheet of individual pixel filters) thereby reducing cost, improvingsignal strength and eliminating the pixel reduction barrier.

In some embodiments, once the integrated circuit die with the sensorarrays (and possibly one or more portions of the processor) have beenassembled, the assembly is in the form of a hermetically sealed device.Consequently, such device does not need a “package” and as such, ifdesired, can be mounted directly to a circuit board which in someembodiments saves part cost and/or manufacturing costs. As stated above,the method and apparatus of the present invention is not limited to usein digital camera systems but rather may be used in any type of systemincluding but not limited to any type of information system.

It should be understood that the features disclosed herein can be usedin any combination.

Note that, except where otherwise stated, terms such as, for example,“comprises”, “has”, “includes”, and all forms thereof, are consideredopen-ended, so as not to preclude additional elements and/or features.Further, except where otherwise stated, terms such as, for example, “inresponse to” and “based on” mean “in response at least to” and “based atleast on”, respectively, so as not to preclude being responsive toand/or based on, more than one thing.

As used herein identifying, determining, and generating includesidentifying, determining, and generating, respectively, in any way,including, but not limited to, computing, accessing stored data and/ormapping (e.g., in a look up table) and/or combinations thereof.

While there have been shown and described various embodiments, it willbe understood by those skilled in the art that the present invention isnot limited to such embodiments, which have been presented by way ofexample only, and various changes and modifications may be made withoutdeparting from the scope of the invention.

1. A digital camera comprising: a plurality of arrays of photodetectors, including: a first array of photo detectors to sample anintensity of light; and a second array of photo detectors to sample anintensity of light; signal processing circuitry, coupled to the firstand second arrays of photo detectors, to generate a composite imageusing (i) data which is representative of the intensity of light sampledby the first array of photo detectors, and (ii) data which isrepresentative of the intensity of light sampled by the second array ofphoto detectors; and wherein the first array of photo detectors, thesecond array of photo detectors, and the signal processing circuitry areintegrated on or in the same semiconductor substrate.
 2. The digitalcamera of claim 1 wherein: the first array of photo detectors sample anintensity of light of a first wavelength; and the second array of photodetectors sample an intensity of light of a second wavelength.
 3. Thedigital camera of claim 2 further including a third array of photodetectors to sample the intensity of light of a third wavelength, andwherein the signal processing circuitry is coupled to the third array ofphoto detectors and generates a composite image using (i) data which isrepresentative of the intensity of light sampled by the first array ofphoto detectors, (ii) data which is representative of the intensity oflight sampled by the second array of photo detectors, and (ii) datawhich is representative of the intensity of light sampled by the thirdarray of photo detectors.
 4. The digital camera of claim 3 wherein thefirst, second and third arrays of photo detectors are relativelyarranged in a triangular configuration.
 5. The digital camera of claim 4wherein the first array of photo detectors, the second array of photodetectors, the third array of photo detectors, and the signal processingcircuitry are integrated on the same semiconductor substrate.
 6. Thedigital camera of claim 5 wherein the first wavelength is associatedwith a first color, the second wavelength is associated with a secondcolor and the third wavelength is associated with a third color.
 7. Thedigital camera of claim 3 wherein: the first array of photo detectorssample the intensity of light of the first wavelength for a firstintegration time; the second array of photo detectors sample theintensity of light of the second wavelength for a second integrationtime; and the third array of photo detectors sample the intensity oflight of the third wavelength for a third integration time.
 8. Thedigital camera of claim 3 wherein the first, second and third arrays ofphoto detectors are relatively arranged in an isosceles, obtuse, acuteor a right triangular configuration.
 9. The digital camera of claim 2wherein the first wavelength is associated with a first color and thesecond wavelength is associated with a second color.
 10. The digitalcamera of claim 1 wherein: the first array of photo detectors sample theintensity of light for a first integration time; and the second array ofphoto detectors sample the intensity of light for a second integrationtime.
 11. The digital camera of claim 1 wherein: each photo detector ofthe first array includes a semiconductor portion at which the intensityof light is sampled; and each photo detector of the second arrayincludes a semiconductor portion at which the intensity of light issampled and wherein the semiconductor portion of each photo detector ofthe first array is located at a different depth, relative to a surfaceof each of the photo detectors, from than semiconductor portion of eachphoto detector of the second array.
 12. The digital camera of claim 1wherein the first array of photo detectors and the second array of photodetectors are disposed in the same image plane.
 13. The digital cameraof claim 1 further including a first lens disposed in and associatedwith an optical path of the first array of photo detectors and a secondlens disposed in and associated with an optical path of the second arrayof photo detectors.
 14. The digital camera of claim 13 further includinga substantially uniform color filter sheet disposed in the optical pathof the first array of photo detectors.
 15. The digital camera of claim 1further including a first colored lens disposed in and associated withan optical path of the first array of photo detectors.
 16. The digitalcamera of claim 1 further including a first lens disposed in andassociated with an optical path of the first array of photo detectors,wherein: the first lens passes light of a first wavelength and filterslight of a second wavelength; the first array of photo detectors samplean intensity of light of a first wavelength; and the second array ofphoto detectors sample an intensity of light of a second wavelength. 17.The digital camera of claim 1 wherein: the first array of photodetectors sample an intensity of light of a first wavelength and anintensity of light of a second wavelength; the second array of photodetectors sample an intensity of light of a third wavelength; and thefirst wavelength is associated with a first color, the second wavelengthis associated with a second color and the third wavelength is associatedwith a third color.
 18. The digital camera of claim 17 wherein: eachphoto detector of the first array includes a first semiconductor portionat which the intensity of light of the first wavelength is sampled and asecond semiconductor portion at which the intensity of light of thesecond wavelength is sampled; each photo detector of the second arrayincludes a semiconductor portion at which the intensity of light of thethird wavelength is sampled; and the first and second semiconductorportions of each photo detector of the first array are located at adifferent depth, relative to each other and to a surface of each of thephoto detectors from the semiconductor portion of each photo detector ofthe second array.
 19. The digital camera of claim 17 further including afirst lens disposed in and associated with an optical path of the firstarray of photo detectors and a second lens disposed in and associatedwith an optical path of the second array of photo detectors wherein thefirst lens passes light of the first and second wavelengths and filterslight of the third wavelength.
 20. The digital camera of claim 17further including an optical filter disposed in and associated with anoptical path of the first array of photo detectors wherein the opticalfilter passes light of the first and second wavelengths and filterslight of the third wavelength.
 21. The digital camera of claim 17wherein: the first array of photo detectors sample the intensity oflight of the first wavelength for a first integration time; the firstarray of photo detectors sample the intensity of light of the secondwavelength for a second integration time; and the second array of photodetectors sample the intensity of light of the third wavelength for athird integration time.
 22. The digital camera of claim 1 wherein thesignal processing circuitry generates: a first image using data which isrepresentative of the intensity of light sampled by the first array ofphoto detectors, and a second image using data which is representativeof the intensity of light sampled by the second array of photodetectors.
 23. The digital camera of claim 22 wherein the signalprocessing circuitry generates the composite image using the first imageand the second image.
 24. The digital camera of claim 1 furtherincluding a memory to store (i) data which is representative of theintensity of light sampled by the first array of photo detectors, and(ii) data which is representative of the intensity of light sampled bythe second array of photo detectors.
 25. The digital camera of claim 24wherein the memory, the first array of photo detectors, the second arrayof photo detectors, and the signal processing circuitry are integratedon or in the same semiconductor substrate.
 26. The digital camera ofclaim 25 further including timing and control logic that to providetiming and control information to the signal processing circuitry, thefirst array of photo detectors and/or the second array of photodetectors.
 27. The digital camera of claim 24 further includingcommunication circuitry to output data which is representative of thecomposite image.
 28. The digital camera of claim 27 wherein thecommunication circuitry includes at least one of wireless, wired oroptical communication circuitry.
 29. The digital camera of claim 24wherein the communication circuitry, the memory, the first array ofphoto detectors, the second array of photo detectors, and the signalprocessing circuitry are integrated on or in the same semiconductorsubstrate.
 30. The digital camera of claim 1 wherein the first array ofphoto detectors includes a first surface area and the second array ofphoto detectors includes a second surface area wherein the first surfacearea is different from the second surface area.
 31. The digital cameraof claim 30 wherein the photo detectors of the first array include afirst active surface area and the photo detectors of the second arrayinclude a second active surface area wherein the first active surfacearea is different from the second active surface area.
 32. The digitalcamera of claim 1 wherein the first array of photo detectors includes afirst surface area and the second array of photo detectors includes asecond surface area wherein the first surface area is substantially thesame as the second surface area.
 33. The digital camera of claim 32wherein the photo detectors of the first array include a first activesurface area and the photo detectors of the second array include asecond active surface area wherein the first active surface area isdifferent from the second active surface area.
 34. A digital cameracomprising: a plurality of arrays of photo detectors, including: a firstarray of photo detectors to sample an intensity of light of a firstwavelength; and a second array of photo detectors to sample an intensityof light of a second wavelength; a first lens disposed in an opticalpath of the first array of photo detectors wherein the first lensincludes a predetermined optical response to the light of the firstwavelength; a second lens disposed in with an optical path of the secondarray of photo detectors wherein the second lens includes apredetermined optical response to the light of the second wavelength;signal processing circuitry, coupled to the first and second arrays ofphoto detectors, to generate a composite image using (i) data which isrepresentative of the intensity of light sampled by the first array ofphoto detectors, and (ii) data which is representative of the intensityof light sampled by the second array of photo detectors; and wherein thefirst array of photo detectors, the second array of photo detectors, andthe signal processing circuitry are integrated on or in the samesemiconductor substrate.
 35. The digital camera of claim 34 wherein: thefirst lens passes light of the first wavelength onto an image plane ofthe photo detectors of the first array; and the second lens passes lightof the second wavelength onto an image plane of the photo detectors ofthe second array.
 36. The digital camera of claim 35 wherein: the firstlens filters light of the second wavelength; and the second lens filterslight of the first wavelength.
 37. The digital camera of claim 34further including: a third array of photo detectors to sample theintensity of light of a third wavelength; a third lens disposed in withan optical path of the third array of photo detectors wherein the thirdlens includes a predetermined optical response to the light of the thirdwavelength; and and wherein the signal processing circuitry is coupledto the third array of photo detectors and generates a composite imageusing (i) data which is representative of the intensity of light sampledby the first array of photo detectors, (ii) data which is representativeof the intensity of light sampled by the second array of photodetectors, and (ii) data which is representative of the intensity oflight sampled by the third array of photo detectors.
 38. The digitalcamera of claim 37 wherein: the first lens filters light of the secondand third wavelengths; the second lens filters light of the first andthird wavelengths; and the third lens filters light of the first andsecond wavelengths.
 39. The digital camera of claim 37 wherein thefirst, second and third arrays of photo detectors are relativelyarranged in a triangular configuration.
 40. The digital camera of claim39 wherein the first array of photo detectors, the second array of photodetectors, the third array of photo detectors, and the signal processingcircuitry are integrated on the same semiconductor substrate.
 41. Thedigital camera of claim 40 wherein the first wavelength is associatedwith a first color, the second wavelength is associated with a secondcolor and the third wavelength is associated with a third color.
 42. Thedigital camera of claim 37 wherein: the first array of photo detectorssample the intensity of light of the first wavelength for a firstintegration time; the second array of photo detectors sample theintensity of light of the second wavelength for a second integrationtime; and the third array of photo detectors sample the intensity oflight of the third wavelength for a third integration time.
 43. Thedigital camera of claim 34 wherein the first wavelength is associatedwith a first color and the second wavelength is associated with a secondcolor.
 44. The digital camera of claim 43 wherein: the first array ofphoto detectors sample the intensity of light for a first integrationtime; and the second array of photo detectors sample the intensity oflight for a second integration time.
 45. The digital camera of claim 34further including a housing, wherein the first and second lenses, firstand second arrays of photo detectors, and the signal processingcircuitry are attached to the housing, and wherein the first and secondlenses are independently positionable relative to the associated arrayof photo detectors.
 46. The digital camera of claim 34 wherein: thefirst array of photo detectors sample an intensity of light of the firstwavelength and an intensity of light of a third wavelength; the secondarray of photo detectors sample an intensity of light of a secondwavelength; and the first wavelength is associated with a first color,the second wavelength is associated with a second color and the thirdwavelength is associated with a third color.
 47. The digital camera ofclaim 46 wherein: each photo detector of the first array includes afirst semiconductor portion at which the intensity of light of the firstwavelength is sampled and a second semiconductor portion at which theintensity of light of the third wavelength is sampled; each photodetector of the second array includes a semiconductor portion at whichthe intensity of light of the second wavelength is sampled; and thefirst and second semiconductor portions of each photo detector of thefirst array are located at a different depth, relative to each other andto a surface of each of the photo detectors from the semiconductorportion of each photo detector of the second array.
 48. The digitalcamera of claim 46 wherein the first lens passes light of the first andthird wavelengths and filters light of a second wavelength.
 49. Thedigital camera of claim 46 further including an optical filter disposedin and associated with an optical path of the first array of photodetectors wherein the optical filter passes light of the first and thirdwavelengths and filters light of the second wavelength.
 50. The digitalcamera of claim 46 wherein: the first array of photo detectors samplethe intensity of light of the first wavelength for a first integrationtime; the second array of photo detectors sample the intensity of lightof the third wavelength for a second integration time; and the firstarray of photo detectors sample the intensity of light of the thirdwavelength for a third integration time.
 51. The digital camera of claim34 wherein the signal processing circuitry generates: a first imageusing data which is representative of the intensity of light sampled bythe first array of photo detectors, and a second image using data whichis representative of the intensity of light sampled by the second arrayof photo detectors.
 52. The digital camera of claim 51 wherein thesignal processing circuitry generates the composite image using thefirst image and the second image.
 53. The digital camera of claim 34further including a memory to store (i) data which is representative ofthe intensity of light sampled by the first array of photo detectors,and (ii) data which is representative of the intensity of light sampledby the second array of photo detectors.
 54. The digital camera of claim53 wherein the memory, the first array of photo detectors, the secondarray of photo detectors, and the signal processing circuitry areintegrated on or in the same semiconductor substrate.
 55. The digitalcamera of claim 53 further including communication circuitry to outputdata which is representative of the composite image.
 56. The digitalcamera of claim 55 wherein the communication circuitry includes at leastone of wireless, wired or optical communication circuitry.
 57. Thedigital camera of claim 53 wherein the communication circuitry, thememory, the first array of photo detectors, the second array of photodetectors, and the signal processing circuitry are integrated on or inthe same semiconductor substrate.
 58. The digital camera of claim 34wherein the signal processing circuitry includes first signal processingcircuitry and second signal processing circuitry wherein the firstsignal processing circuitry is coupled to and associated with the firstarray of photo detectors and second signal processing circuitry iscoupled to and associated with the second array of photo detectors. 59.The digital camera of claim 34 wherein the signal processing circuitryincludes first analog signal logic and second analog signal logicwherein the first analog signal logic is coupled to and associated withthe first array of photo detectors and second analog signal logic iscoupled to and associated with the second array of photo detectors. 60.The digital camera of claim 34 wherein the signal processing circuitryincludes first black level logic and second black level logic whereinthe first black level logic is coupled to and associated with the firstarray of photo detectors and second black level logic is coupled to andassociated with the second array of photo detectors.
 61. The digitalcamera of claim 34 wherein the signal processing circuitry includesfirst exposure control circuitry and second exposure control circuitrywherein the first exposure control circuitry is coupled to andassociated with the first array of photo detectors and second exposurecontrol circuitry is coupled to and associated with the second array ofphoto detectors.
 62. The digital camera of claim 34 further including aframe, wherein the first and second arrays of photo detectors, thesignal processing circuitry, and the first and second lenses are fixedto the frame.
 63. The digital camera of claim 34 further includingtiming and control logic that to provide timing and control informationto the signal processing circuitry, the first array of photo detectorsand/or the second array of photo detectors.
 64. The digital camera ofclaim 34 wherein the first array of photo detectors includes a firstsurface area and the second array of photo detectors includes a secondsurface area wherein the first surface area is different from the secondsurface area.
 65. The digital camera of claim 64 wherein the photodetectors of the first array include a first active surface area and thephoto detectors of the second array include a second active surface areawherein the first active surface area is different from the secondactive surface area.
 66. The digital camera of claim 34 wherein thefirst array of photo detectors includes a first surface area and thesecond array of photo detectors includes a second surface area whereinthe first surface area is substantially the same as the second surfacearea.
 67. The digital camera of claim 66 wherein the photo detectors ofthe first array include a first active surface area and the photodetectors of the second array include a second active surface areawherein the first active surface area is different from the secondactive surface area.